AD-A216  381 


r'  f  -  .  -  ;-rj&T^ 

X  i'  j  4.X  iAJ'?  i 


RADC-TR-89-305 
Final  Technical  Report 

November  1989 


3D  APPLICATION  STUDY 


I  IT  Research  Institute 


Russel  D.  Mikel 


APPROVED  FOR  PUBUC  RELEASE;  DISTRIBUTION  UNLIMITED 


ROME  AIR  DEVELOPMENT  CENTER 
Air  Force  Systems  Command 
Griffiss  Air  Force  Base,  NY  13441-5700 


90  01  03  055 


This  report  has  been  reviewed  by  the  RADC  Public  Affairs  Division  (PA) 
and  is  releasable  to  the  National  Technical  Information  Services  (NTIS)  At 
NTIS  it  will  be  releasable  to  the  general  public,  including  foreign  nations. 

RADC-TR-89-305  has  been  reviewed  and  is  approved  for  publication. 

I 

RICHARD  T.  SLAVINSKI 
Project  Engineer 


* 

APPROVED :  ^ 

RAYMOND  P.  URTZ,  JR. 

Technical  Director 
Directorate  of  Command  &  Control 


Directorate  of  Plans  &  Programs 


If  your  address  has  changed  or  if  you  wish  to  be  removed  from  the  RADC 
mailing  list,  or  if  the  addressee  is  no  longer  employed  by  your 
organization,  please  notify  RADC  (  COE  )  Griffiss  AFB  NY  13441-5700. 
This  will  assist  us  in  maintaining  a  current  mailing  list. 

Do  not  return  copies  of  this  report  unless  contractual  obligations  or 
notices  on  a  specific  document  require  that  it  be  returned. 


^fflaMeaHHHaiBBKiBBaa 


REPORT  DOCUMENTATION  PAGE 

Form  Approved 

OMB  NO.  07044188 

1*.  REPORT  SECURITY  CLASSIFICATION 

UNCLASSIFIED 

(RESTRICTIVE  MARKINGS 

4.  PERFORMING  ORGANIZATION  REPORT  NUMBER(S) 
N/A 


6*  NAME  OF  PERFORMING  ORGANIZATION 
I IT  Research  Institute 


6c  ADDRESS  (City,  Statt,  and  ZIP  Code) 
Beeches  Technical  Campus 
Route  26  North 
Rome  NY  13440-2069 


8a.  NAME  OF  FUNOING/ SPONSORING 
ORGANIZATION 

Rome  Air  Development  Center 


8c  ADDRESS  (City,  Statt,  and  ZIP  Code) 
Griffiss  AFB  NY  13441-5700 


1 1 .  TITLE  (Include  Security  Classification) 
3D  APPLICATION  STUDY 


12.  PERSONAL  AUTHOR(S) 
Russel  D.  Mikel 


6b.  OFFICE  SYMBOL 
(If  applicable) 


8b.  OFFICE  SYMBOL 
(If  applicable) 
COE 


3.  DISTRIBUTION /AVAILABILITY  OF  REPORT 
Approved  for  public  release; 
distribution  unlimited. 


S.  MONITORING  ORGANIZATION  REPORT  NUMBER(S) 
RADC-TR-89-305 


7a.  NAME  OF  MONITORING  ORGANIZATION 
Rome  Air  Development  Center  (COE) 


7b.  ADDRESS  (City,  Statt,  and  ZIP  Code) 
Griffiss  AFB  NY  13441-5700 


9  PROCUREMENT  INSTRUMENT  IDENTIFICATION  NUMBER 


F30602-87-D-0094,  Task  3 


10.  SOURCE  OF  FUNDING  NUMBERS 


PROGRAM 
ELEMENT  NO. 

62702F 


PROJECT- 

TASK 

NO 

NO 

5581 

QC 

13a.  TYPE  OF  REPORT 
Final 


16.  SUPPLEMENTARY  NOTATION 
N/A 


13b.  TIME  COVEREO 
FROM  Feb  88  to  88 


14.  DATE  OF  REPORT 
November 


ar.  Month.  Day)  115.  PAGE  COUNT 

59  *  58 


COSATI  COOES  18.  SUBJECT  TERMS  (Continue  on  reverse  If  necessary  and  identify  by  block  number) 

GROUP  I  SUB-GROUP  3D  Display 


19.  ABSTRACT  ( Continue  on  reverse  If  necessary  and  identify  by  block  number) 

This  effort  considered  ways  in  which  the  application  of  three  dimensional  display 
technology  could  improve  task  performance  in  the  battle  management  arena.  A  subset  of 
a  typical  Air  Force  C3I  battle  management  application  was  analyzed  to  evaluate  how  3D 
display  technology  could  aid  in  task  performance.  Appropriate  C  I  applications  were 
studied  to  determine  the  categories  of  problems  that  might  benefit  most  from  anticipated 
3D  capabilities.  Various  approaches  for  the  generation  of  three  dimensional  Images  were 
investigated  in  order  to  determine  their  potential  applicability  to  the  identified 
problem  areas.  Different  technologies  and  techniques  for  3D  image  generation  were 
examined,  and  the  feasibility  of  their  application  was  assessed.  Finally,  a  demonstration 
of  the  use  of  3D  in  support  of  an  approved  application  was  developed.  In  specific,  3D 
benefits  were  Identified,  commercially  available  equipment  was  evaluate^  and  acquired,  and 
a  demonstration  system  was  developed  to  illustrate  the  use  of  3D  in  a  C  I  application. 

This  report  presents  the  results  of  this  study  effort.  ^  j./ 


20.  DISTRIBUTION /AVAILABILITY  OF  ABSTRACT 
□  UNCLASSIFIEOAJNLIMITED  &  SAME  AS  RPT. 


22*.  NAME  OF  RESPONSIBLE  INDIVIDUAL 
Richard  T.  Slavinski 


□  OTIC  USERS 


21.  ABSTRACT  SECURITY  CLASSIFICATIO 
UNCLASSIFIED 


OO  Form  1473,  JUN  86 


Previous  editions  are  obsolete. 


FOREWORD 

This  is  the  Final  Technical  Report,  CDRL  No.  C003,  for  Task  3  under  contract 
F30602-87-D-  0094.  This  contract  is  with  IIT  Research  Institute  (IITRI)  and 
sponsored  by  Rome  Air  Development  Center.  The  work  was  performed  by  IITRI, 
Rome,  NY;  Honeywell  SRC,  Minneapolis,  MN;  Merit  Technology,  Plano,  TX;  and  SRI 
International,  Menlo  Park,  CA. 

I 


Accession  for 


HTIS  GRAAI 
DTIC  TAB 
Unannounced 
Justification. 


□ 

□ 


Distribution/ 


Availability  Codes 


Dlst 


Avail  and/or 
Special 


3D  APPLICATION  STUDY 


Table  of  Contents 

Page 

Executive  Summary  .  iii 

1.0  Introduction  .  1 

2.0  Depth  Perception  .  1 

2.1  Depth  Perception  in  2D  vs.  3D .  1 

2.2  The  3D  Display  of  Information .  3 

3.0  3D  in  C3I  Applications  .  8 

3.1  Candidate  Applications  and  Data  Types  .  8 

3.2  Tentative  Benefits  to  Battle  Management  .  12 

3.3  Specific  Battle  Management  Applications  .  13 

4.0  3D  Display  Techniques .  15 

4.1  Stereoscopic  Techniques  .  16 

4.2  Volumetric  Techniques  .  24 

4.3  Continuing  Developments  .  30 

5.0  SCENARIO  3D  Demonstration  System  .  31 

5.1  Intent  of  the  Demonstration .  31 

5.2  Implementation  of  SCENARIO  .  31 

5.3  Review  of  the  Demonstration  .  35 

6.0  Conclusions .  37 

7.0  References  .  j9 


1 


30  APPLICATION  STUDY  FINAL  REPORT 


Executive  Suninary 

Introduction 

The  purpose  of  this  effort  was  to  consider  ways  in  which  the  application 
of  three  dimensional  display  technology  could  improve  task  performance  in  the 
battle  management  arena.  Three  tasks  comprised  this  effort.  The  first  was 
the  analysis  of  a  subset  of  typical  Air  Force  C3I  battle  management 
applications  to  evaluate  how  30  display  technology  could  aid  in  task 
performance.  Toward  this  end,  appropriate  C3I  applications  were  studied  to 
determine  the  categories  of  problems  that  might  benefit  most  from  anticipated 
30  capabilities.  The  second  task  was  the  investigation  of  various  approaches 
to  the  generation  of  three  dimensional  images  in  order  to  determine  their 
potential  applicability  to  the  problem  areas  identified  in  Task  1.  Different 
technologies  and  techniques  for  30  image  generation  were  examined,  and  the 
feasibility  of  their  application  to  the  problem  areas  of  interest  was 
assessed.  The  third  task  called  for  the  demonstration  of  the  use  of  3D  in 
support  of  an  approved  application.  In  support  of  this  task,  specific  3D 
benefits  were  identified  as  providing  improved  capabilities,  commercially 
available  equipment  was  evaluated  and  acquired,  and  a  demonstration  system 
was  developed  that  would  illustrate  the  use  of  3D  in  a  C3I  application.  This 
report  presents  the  results  of  the  study  phase  of  the  effort.  Delivered  as  a 
separate  document  with  the  Final  Report  is  the  User's/Operator's  Manual  for 
the  demonstration  software. 

Depth  Perception 

Most  people  are  familiar  with  monocular  depth  cues,  such  as  nearer 
objects  occluding  more  distant  ones,  linear  perspective  and  shadows  cast  by 
lighting.  The  3D  display  of  images  requires  a  binocular  perception  of  the 
image  and  allows  the  viewer  to  synthesize  a  three  dimensional  image  of  the 
scene  that  more  accurately  represents  the  relative  position  and  size  of 
objects  in  the  scene.  The  primary  benefit  to  be  gained  from  the  display  of 
information  in  three  dimensions  is  that  it  allows  the  user  to  perceive  these 
relationships  rather  than  require  him  to  cognitively  translate  tabular  or  two 
dimensional  data  into  an  internal  representation  with  3D  characteristics. 
These  perceptual  benefits  are  of  four  major  types:  a  more  rapid  and  accurate 
grasp  of  spatial  features,  improved  scaling  of  physical  attributes  such  as 
distance  and  velocity,  disambiguation/decluttering  of  graphic  and  textual 
information  on  a  display,  and  the  ability  to  interactively  simulate  abnormal 
sensorimotor  relationships  such  as  those  found  in  microgravity. 

There  is  also  evidence  for  potential  cognitive  benefits  of  3D  displays. 
The  ease  and  effectiveness  with  which  the  mind  processes  information  relates 
directly  to  the  form  In  which  the  information  was  presented.  The  additional 
dimension  provided  by  3D  displays  allows  trainers  at  the  Interceptor  Weapons 
School,  for  example,  to  clearly  demonstrate  the  effects  of  the  timing  of  a 
turn  on  the  geometry  of  the  intercept.  3D  displays  can  also  facilitate 
information  retention  and  retrieval.  Another  promising  area  In  terms  of 
cognitive  benefits  is  that  of  anticipation/expectation.  By  providing 
displays  of  synthesized  data  that  relate  directly  to  the  pilot's  immediate 
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decision  making  process,  it  is  possible  to  reduce  the  information  load  on  the 
pilot. 

3D  in  C3I  Applications 

In  light  of  the  possible  benefits  that  3D  displays  could  provide,  an 
analysis  was  conducted  to  determine  which  C3I  applications  might  expect  to 
see  the  greatest  improvement  in  task  performance  as  a  result  of  applying  3D 
display  technology.  This  analysis  considered  the  types  of  data  appropriate 
for  3D  displays  and  specific  unique  benefits  that  3D  might  provide,  then 
mapped  them  to  specific  battle  management  applications. 

Data  suitable  for  representation  in  three  dimensions  falls  into  either  of 
two  categories.  The  first  and  the  most  obvious  is  that  of  data  with 
inherently  spatial  characteristics.  This  includes  both  objects  that  may 
exist  in  space,  such  as  aircraft  and  terrain,  as  well  as  more  abstract  data, 
like  a  volume  representing  radar  coverage  at  a  particular  sensitivity.  The 
other  type  of  data  appropriate  for  display  in  three  dimensions  lacks  spatial 
features,  but  uses  the  third  dimension  for  other  values.  Various  forms  cf 
networks  showing  relationships  between  complex  sets  of  variables  are  an 
example  of  the  growing  field  of  data  visualization. 

Three  tentative  benefits  resulting  from  the  use  of  3D  displays  for  C3I 
applications  were  identified  during  the  course  of  this  study.  Decluttering 
an  image  allows  a  user  to  view  a  large  amount  of  information  on  a  screen  more 
easily  without  reducing  the  amount  of  data  displayed.  Information  presented 
at  different  depth  planes  enables  the  viewer  to  selectively  focus  on  data 
with  less  distraction.  The  second  benefit  was  the  ability  to  perceive 
irregularly  shaped  volumes.  The  form  of  regular  geometric  shapes  can  be 
inferred  from  2D  perspective  representations,  but  irregularly  shaped  volumes, 
such  as  those  depicting  realistic  electronic  envelopes,  cannot  be  adequately 
perceived  by  viewing  the  image  imposed  on  a  two  dimensional  surface. 
Finally,  the  perception  of  3D  images  over  time,  as  in  a  battlefield 
simulation,  essentially  allows  the  user  to  perceive  a  fourth  dimension,  that 
of  time.  This  capability  results  in  a  more  rapid  and  accurate  estimation  of 
speed,  velocity  and  the  time  at  which  an  object  will  reach  a  particular  point 
in  space  or  intercept  another  object. 

Considering  the  types  of  data  that  lend  themselves  to  3D  and  the  benefits 
that  3D  can  provide,  two  primary  C3I  functions  were  identified  as  the  most 
promising  for  the  application  of  3D  display  technology.  Situation 
assessment,  though  characteristically  reliant  on  static  displays  of 
information,  provides  several  opportunities  to  make  effective  use  of  two  3D 
features:  movement  over  time  and  the  representation  of  data  by  volumes.  The 
historical  movement  of  military  and  natural  forces  within  the  geographic  area 
of  interest  could  provide  insights  into  the  effects  of  terrain  on  movement 
and  imparting  a  sense  of  distance  between  forces  moving  in  three  dimensional 
space.  Similarly,  force  movement  could  be  projected  into  the  relatively  near 
future  to  determine  the  possible  consequences  of  current  activity.  More 
appropriate  to  situation  assessment  activities  would  be  the  use  of  3D  to 
provide  a  spatial  view  of  electronic  warfare  equipment  characteristics  such 
as  coverage,  effective  radiated  power,  and  vulnerability  to  specific 
countermeasures.  In  the  area  of  operational  mission  planning,  3D  provides 
particularly  appropriate  capabilities.  The  visual  modeling  of  mission 
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execution  from  the  viewpoint  of  the  mission  planner  is  perhaps  the  most 
obvious  application.  Another  possibility,  and  a  potentially  more  valuable 
one,  is  the  simulation  of  a  mission  from  the  perspective  of  the  mission 
pilot.  Because  of  the  scale  and  the  real-time  nature  of  the  pilot's 
interaction  with  his  environment,  this  is  one  of  the  most  compelling 
candidate  applications  for  enhancement  by  3D  display  technology. 

3D  Display  Techniques 

Three  dimensional  display  techniques  can  be  grouped  into  two  primary 
categories:  stereoscopic  and  volumetric.  Stereoscopic  approaches  involve 
the  presentation  of  two  slightly  offset  images  of  a  view  to  each  of  the 
viewer's  eyes.  This  offset  duplicates  the  different  perspective  views  of  the 
two  eyes,  providing  a  perception  of  binocular  depth.  The  problem  associated 
with  providing  stereoscopic  views  is  the  manner  in  which  the  visual  access  to 
each  eye  is  restricted  to  the  appropriate  image.  Most  systems  require  the 
user  to  wear  some  form  of  spectacles  (colored  or  polarized)  which  separate 
the  two  images.  Some,  like  the  parallax  panoramagram,  are  autostereoscopic 
and  require  no  user-worn  aids  to  separate  the  views  to  each  eye.  Of  the 
stereo  systems  examined,  the  field  sequential  display  with  image  separation 
at  the  source  (requiring  the  viewer  to  wear  passive  polarized  spectacles) 
represents  the  most  mature  technology  and  is  commercially  available  for  use 
with  computer  displays.  Given  an  appropriately  powerful  graphics  computer 
system,  the  addition  of  stereo  capability  can  provide  sufficient  processing 
speed,  display  resolution  and  color  for  many  battle  management  applications, 
particularly  in  the  area  of  mission  planning.  The  need  to  wear  passive 
glasses  is  a  minor  inconvenience,  and  the  use  of  polarized  filters  reduces 
available  brightness  significantly.  Like  all  stereoscopic  systems,  the 
technique  fails  to  preserve  full  motion  parallax.  When  the  viewer  moves  his 
head  to  the  right,  for  example,  the  images  do  not  reveal  previously  hidden 
surfaces  as  occurs  in  true  three-space.  Rather,  they  appear  to  move  to  the 
right  as  well.  This  effect  can  be  a  source  of  annoyance  or  confusion  to  the 
viewer. 

Volumetric  techniques  attempt  to  create  a  three  dimensional  image  as 
opposed  to  presenting  two  perspective  views  of  a  particular  scene.  Because 
of  the  spatial  nature  of  their  images,  most  volumetric  systems  have  the 
advantage  of  being  viewable  by  multiple  observers  without  the  need  for  user- 
worn  aids.  Holography  is  one  technology  that  has  generated  a  great  deal  of 
interest.  Unfortunately,  due  to  the  processing  time  required  for  the 
production  of  solely  static  images,  real-time  applications  are  not  expected 
to  be  available  in  the  foreseeable  future.  A  volumetric  system  that  has  been 
commercially  iriarketed,  however.  Is  based  on  a  varifocal  mirror  technique. 
The  SpaceGraph  system  uses  a  vibrating  mirror  to  reflect  a  two  dimensional 
computer  screen  image.  By  synchronizing  the  display  of  points  and  lines  with 
the  oscillating  silvered  membrane,  SpaceGraph  essentially  draws  a  three 
dimensional  image  in  space.  The  result  is  a  very  compelling  image.  No  aids 
are  required  to  be  worn  by  the  viewer,  and  the  number  of  viewers  is  limited 
primarily  by  the  relatively  small  display  space  of  the  system.  Unlike  other 
available  systems,  this  technique  preserves  full  head-motion  parallax.  The 
most  significant  drawbacks  of  SpaceGraph  are  the  lack  of  color  capability  due 
to  current  phosphor  extinction  rates  and  the  limitations  on  the  number  of 
pixels  that  can  be  addressed  when  the  mirror  is  in  a  particular  position  (in 
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turn,  limiting  the  complexity  of  displayed  images).  If  some  of  these 
difficulties  can  be  overcome,  vibrating  mirror  technology  offers  a  potential 
alternative  to  stereoscopic  display  systems. 

SCENARIO  30  Demonstration  System 

In  addition  to  the  completion  of  this  study,  this  effort  required  the 
development  of  a  demonstration  using  a  three  dimensional  presentation  in 
support  of  a  sample  Air  Force  C3I  battle  management  function.  It  was  decided 
that  the  system  should  highlight  the  three  situations  in  which  3D  was  felt  to 
provide  additional  unique  capabilities.  The  first  of  these  is  decluttering, 
in  which  elements  of  textual  and  graphical  information  displayed  close 
together  might  be  disambiguated  by  viewing  them  in  their  respective  depth 
planes.  An  example  of  this  would  be  in  air  space  management,  where 
information  on  multiple  moving  and  overlapping  objects  must  be  discernable. 
The  second  was  in  the  display  of  irregularly  shaped  objects  and  their 
intersection.  While  the  viewer  can  imagine  the  shapes  of  regular  geometric 
forms  displayed  on  a  2D  screen,  the  shapes  of  such  forms  as  electronic  threat 
envelopes  can  be  very  difficult  to  infer  from  a  2D  representation.  The  third 
situation  considered  appropriate  for  a  3D  display  was  the  depiction  of  3D 
data  as  it  changes  and  moves  over  time,  providing  the  viewer  with  the  ability 
to  project  the  time  at  which  objects  will  reach  a  certain  point  or  intersect 
with  other  objects.  An  outline  was  developed  to  exhibit  3D  capabilities  in  a 
tactical  air  support  situation  in  which  the  user  is  tasked  to  create  a  set  of 
waypoints  from  a  friendly  airfield  to  an  enemy  target  with  a  minimum  exposure 
to  threats,  such  as  enemy  radars  and  contaminated  areas. 

Hardware  and  software  options  for  supporting  the  demonstration  were 
considered.  The  eventual  choice  was  the  purchase  of  a  Silicon  Graphics 
graphics  workstation  with  a  StereoGraphics  liquid  crystal  shuttering  system. 
The  decision  was  based  largely  on  team  familiarity  with  the  system  and  the 
reasonable  cost  of  systems  used  by  Silicon  Graphics  for  demonstration 
purposes.  Given  the  limited  time  and  budget  for  the  development  of  the 
demonstration,  these  were  both  significant  factors. 

As  implemented,  the  demonstration  program,  called  SCENARIO,  can  use 
either  of  two  terrain  databases  and  provides  the  user  with  the  ability  to 
select  between  a  2D  plan  view  of  the  scene,  a  2-1/2D  perspective  view,  and  a 
3D  stereoscopic  view.  The  user  is  able  to  display  such  features  as 
electronic  threat  domes  and  dispersing  contamination  clouds  while 
interactively  defining  the  route  of  the  aircraft  and  previewing  the  aircraft 
flying  it  in  near  real-time.  The  process  of  developing  and  reviewing  the 
demonstration  system  provided  insights  into  the  use  of  3D  for  operational 
tasks  that  had  not  been  considered  earlier.  The  stereo  system  is  very 
sensitive  to  the  parameters  of  the  user's  vision.  The  ease  with  which 
stereoscopic  depth  is  perceived  varies  markedly  for  different  people  as  well 
as  for  a  single  viewer  during  different  sessions.  By  providing  user  control 
over  the  setting  of  the  viewing  parameters  of  the  stereo  system,  the  SCENARIO 
development  team  felt  that  the  viewer  could  adjust  them  to  maximum  effect 
during  the  session  and  the  software  developer  could  experiment  with  the 
capabilities  of  the  stereo  technology. 
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The  SCENARIO  demonstration  software  provides  the  user  with  an  example  of 
3D  display  technology  applied  to  a  battle  management  application.  Project 
constraints  in  terms  of  team  experience  with  3D  development,  budget  and  time 
restrictions,  and  equipment  performance  capabilities  necessarily  limited 
aspects  of  the  implementation.  In  most  cases,  these  limitations  can  be 
overcome  by  further  development  of  the  system.  In  addition,  the  combined 
hardware  and  software  provide  a  powerful  and  expandable  stereo-equipped  3D 
graphics  workstation  for  continued  exploration  and  development.  The  SCENARIO 
source  code  contains  examples  of  many  aspects  of  stereo  programming  on  the 
system,  and  its  modular  structure  facilitates  both  enhancement  of  SCENARIO  as 
well  as  the  use  of  its  modules  in  other  programs. 

Conclusions 


The  potential  exists  for  achieving  real  and  significant  improvements  in 
the  performance  of  certain  battle  management  activities  as  a  direct  result  of 
the  use  of  three  dimensional  displays.  Primary  anticipated  benefits  are  a 
decluttered  screen  image  providing  greater  accuracy  and  ease  of  use  by  the 
viewer;  an  improved  accuracy  in  perceiving  the  mass,  surface  areas  and 
intersections  of  volumetric  forms;  and  an  increased  insight  into  the 
relationships  of  objects  as  they  interact  over  time.  C3I  battle  management 
functions  within  the  scope  of  this  study  that  can  put  these  enhanced 
capabilities  to  the  best  use  are  in  the  areas  of  situation  assessment  and 
operational  mission  planning,  particularly  applications  which  involve  the 
presentation  of  battlefield  dynamics  for  purposes  of  evaluation.  These 
situations  make  optimal  use  of  the  unique  advantages  of  spatial  and  temporal 
modeling  afforded  by  3D  displays. 

The  ability  to  realize  the  potential  benefits  of  3D  displays  is  currently 
constrained  by  the  state  of  the  available  technology.  Several  techniques 
provide  compelling  3D  images,  but  the  problem  of  human  vision  variability  and 
visual  fatigue  affects  all  3D  methods  to  some  degree.  In  summary,  three 
dimensional  displays  can  provide  the  viewer  with  a  perceptual  reconstruction 
of  three-space  that  is  not  achievable  otherwise.  The  use  of  3D  may  be 
particularly  important  when  monocular  depth  cues  alone  provide  ambiguous 
depth  information.  30  can  resolve  those  ambiguities,  though  depending  upon 
the  technology,  it  may  be  at  the  potential  risk  of  some  viewer  discomfort. 


1.0  INTRODUCTION 


The  purpose  of  this  effort  was  to  consider  ways  in  which  the  application 
of  three  dimensional  display  technology  could  improve  task  performance  in  the 
battle  management  arena.  Three  tasks  comprised  this  effort.  The  first  was 
the  analysis  of  a  subset  of  typical  Air  Force  C3I  battle  management 
applications  to  evaluate  how  3D  display  technology  could  aid  in  task 
performance.  Toward  this  end,  appropriate  C3I  applications  were  studied  to 
determine  the  categories  of  problems  that  might  benefit  most  from  anticipated 
3D  capabilities.  The  second  task  was  the  investigation  of  various  approaches 
to  the  generation  of  three  dimensional  images  in  order  to  determine  their 
potential  applicability  to  the  problem  areas  identified  in  Task  1.  Different 
technologies  and  techniques  for  3D  image  generation  were  examined,  and  the 
feasibility  of  their  application  to  the  problem  areas  of  interest  was 
assessed.  The  third  task  called  for  the  demonstration  of  the  use  of  3D  in 
support  of  an  approved  application.  In  support  of  this  task,  specific  3D 
benefits  were  identified  as  providing  improved  capabilities,  commercially 
available  equipment  was  evaluated  and  acquired,  and  a  demonstration  system 
was  developed  that  would  illustrate  the  use  of  3D  in  a  C3I  application. 

This  report  presents  the  results  of  the  study  phase  of  the  effort. 
Section  2  provides  a  background  discussion  on  the  perception  of  depth  in  two 
and  three  dimensional  images.  Section  3  addresses  candidate  battle 
management  applications  from  the  perspective  of  unique  3D  capabilities. 
Section  4  describes  various  techniques  for  the  generation  of  three 
dimensional  images.  The  discussion  includes  the  advantages  and  disadvantages 
of  the  many  approaches  and  is  intended  to  acquaint  the  reader  with  the  rich 
history  and  diversity  of  attempts  to  create  synthetic  three  dimensional 
images.  Section  5  documents  the  development  and  implementation  of  the 
demonstration  system.  The  demonstration  software  User's/Operator's  Manual  is 
provided  as  a  separate  document.  Section  6  summarizes  the  conclusions  of  the 
task,  derived  both  from  the  study  phase  of  the  effort  and  the  experience 
gained  in  developing  the  demonstration. 


2.0  DEPTH  PERCEPTION 


2.1  Depth  Perception  in  2D  vs.  3D 

Almost  everyone  utilizes  monocular  depth  cues;  they  are  a  fundamental 
visual  function  for  most  animals  and  are  evident  at  very  early  stages  of 
development.  Monocular  depth  cues  include: 

Linear  Perspective:  Euclid's  law;  i.e.,  the  angular  substance  of  an 
object  of  invariant  linear  size  will  be  inversely  proportional  to 
the  distance  of  the  object  from  the  eye. 

Detail  Perspective:  texture  density  also  obeys  Euclid's  law. 

Aerial  Perspective:  Distant  objects  tend  to  have  blurred  contours 
and  appear  more  bluish  In  appearance  than  closer  objects. 

Interposition:  Closer  objects  occlude  farther  objects  along  the 

same  line  of  sight. 
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Light  and  Shade:  e.g.,  shadows  cast  by  a  directional  light  source. 

Motion  Parallax:  Transformations  of  the  visual  field  resulting  from 
movements  of  the  head  relative  to  a  fixed  environment. 

Relative  Size  of  Familiar  Objects:  e.g.,  the  approximate  size  of  a 
car. 

Algorithms  for  handling  true  perspective,  edge  blending,  hidden  line  and 
surface  removal,  shading  and  light  source  location,  contrast  manipulation  and 
animation  in  computer  images  exist  to  generate  the  monocular  depth  cues 
described  above. 

Both  3D  and  perspective  2D  displays  can  present  compelling  depth  cues 
that  enable  the  viewer  to  localize  displayed  objects  in  three-space.  3D 
displays  require  binocular  perception  for  full  appreciation  of  depth,  whereas 
2D  displays  can  be  fully  appreciated  in  monocular  view.  Because  most  depth 
cues  are  monocular  (e.g.,  interposition,  texture  gradients,  perspective),  2D 
displays  can  provide  adequate  depth  rendition  without  the  additional 
technical  requirements  imposed  by  3D  displays.  The  benefit  derived  from 
viewing  3D  as  opposed  to  perspective  2D  displays  is  essentially  the 
difference  between  perceptual  and  cognitive  processing  of  visual  information. 

Because  the  two  eyes  view  the  world  from  different  positions,  the  two 
retinal  images  are  not  identical;  images  of  corresponding  objects  fall  on 
different  retinal  locations  in  the  two  eyes.  Within  the  fusional  limits  of 
the  visual  system,  whenever  disparate  retinal  images  of  the  same  object  are 
presented  to  the  two  eyes,  the  visual  system  uses  the  disparity  information 
to  synthesize  a  three  dimensional  image  of  that  object.  When  disparity 
information  is  absent,  as  when  viewing  a  2D  display,  depth  is  inferred  solely 
from  monocular  depth  cues.  Frequently,  and  especially  in  synthetic  images, 
monocular  depth  cues  can  be  ambiguous;  numerous  visual  illusions  are  based 
upon  this  ambiguity. 

When  information  along  a  third  dimension  is  imposed  onto  a  two- 
dimensional  image  -  even  one  containing  monocular  depth  cues  -  an 
appreciation  of  that  information  may  require  cognitive  processing.  For 
example,  when  position  along  the  Y-axis  is  used  to  encode  some  other 
dimension  (e.g.,  probability  of  detection  by  radar),  a  comparison  of  Y-values 
at  several  X-Z  locations  requires  examination  of  the  relative  distances  of 
the  Y-values  from  the  X-Z  plane.  Because  the  X-Z  plane  must  be  shown  in 
perspective  view,  estimation  of  the  relative  Y-values  requires  cognitive 
transformation  of  the  X-Z  plane,  usually  into  plan  view.  The  same 
information  presenteu  in  a  spatial  form  where  depth  is  visually  perceived 
will  provide  the  viewer  with  immediate  perceptual  information  about  the 
relative  locations  of  the  two  Y-values.  The  transformations  are  inherent  in 
the  pattern  of  retinal  disparity  that  the  visual  system  used  to  generate  the 
depth  percept. 

One  major  advantage  of  3D  displays  is  their  ability  to  disambiguate 
object  locations,  especially  under  relatively  impoverished  viewing 
conditions,  as  when  image  contrast  is  reduced  or  the  images  are  blurred, 
e.g.,  by  vibration.  A  situation  in  which  depth  ambiguity  may  occur  is  when 
objects  are  imbedded  within  a  cluttered  environment.  Here  the  monocular 
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depth  cues  may  fail  to  provide  sufficient  depth  information  to  define  the 
locations  of  specific  objects,  but  information  added  by  the  perceived 
position  of  objects  in  depth  may  be  sufficient  to  permit  discrimination  of 
objects  whose  relative  positions  had  previously  been  ambiguous.  The 
potential  benefits  of  30  in  this  case  would  be  an  increase  in  accuracy  in 
discrimination  tasks,  such  as  identifying  a  particular  aircraft  in  a  dense 
pattern,  and  a  reduction  in  the  error  rate  in  detection  tasks,  such  as  when 
two  aircraft  are  separated  only  on  the  Z-coordinate  axis  of  an  airspace. 

Three  dimensional  displays  that  are  applicable  to  C3  present 
synthetically  generated  representations  of  three-dimensional  images.  These 
representations  do  not  mimic  actual  viewing  conditions  precisely,  and  this 
imprecision  has  important  consequences.  There  are,  for  example,  visual  image 
requirements,  peculiar  to  some  types  of  stereoscopic  displays  (e.g.,  field- 
sequential,  color-anaglyph,  and  polarization  displays  discussed  in  Section  4) 
that  must  be  met  before  stereoscopic  depth  can  be  appreciated.  If  they  are 
not  met,  the  viewer  may  experience  a  delay  in  perceiving  depth  or  may  be 
unable  to  perceive  stereoscopic  depth  at  all.  When  such  visual  requirements 
are  unmet,  the  usual  result  is  diplopia,  an  undesirable  condition  in  which 
the  viewer  perceives  both  monocular  images,  i.e.,  experiences  double  vision. 
Diplopia  degrades  visual  performance  significantly,  and  is  to  be  avoided  at 
all  cost. 

Volumetric  displays,  such  as  the  varifocal-mirror  types,  do  not  use 
artificial  means  to  separate  the  two  images,  so  diplopia  is  not  experienced 
when  viewing  displays  of  these  types.  However,  these,  as  well  as  the  field- 
sequential  types,  can  produce  flicker.  Flicker  can  be  more  than  an 
annoyance;  it  can  contribute  to  another  potential  difficulty  that  viewers  may 
experience  with  stereoscopic  displays,  that  of  "visual  fatigue". 

Visual  fatigue  is  a  poorly  defined,  but  universally  experienced 
phenomenon  that  has  many  causes.  In  the  case  of  stereoscopic  displays,  some 
of  the  causes  of  visual  fatigue  are  flicker,  misalignment  of  the  stereopair, 
and  differences  in  the  size,  color,  or  brightness  of  corresponding  objects  in 
the  stereopair.  Visual  fatigue  can  also  be  attributed  to  conflicting 
accommodation  and  convergence  requirements.  The  accommodation/convergence 
conflict  can  create  serious  problems  for  the  viewer  of  stereoscopic  displays. 
When  viewing  a  display  screen,  the  eyes  accommodate,  i.e.,  focus  on  the 
display  surface  and  the  eyes  converge  so  that  they  fixate  the  same  point  on 
that  surface.  However,  the  stereoscopic  images  presented  on  the  display  are 
not  constrained  to  appear  in  that  surface  plane.  They  may  appear  to  standout 
from  the  surface,  in  which  case  the  eyes  must  accommodate  and  converge  for  a 
closer  object,  or  they  may  appear  to  recede,  which  produces  the  opposite 
accommodative  and  convergence  stimulus.  Thus,  the  eyes  are  driven  by 
conflicting  accommodative/convergence  stimuli  between  the  screen  surface 
distance  and  the  object  image  distance. 

2.2  The  3D  Display  of  Information 

The  display  of  information  in  three  dimensions,  in  contrast  to 
conventional  displays  which  are  limited  to  two  dimensions,  provides  a  variety 
of  benefits  which  can  be  divided  into  two  categories:  those  which  may  be 
labeled  'perceptual',  and  those  which  fall  into  the  'cognitive'  class. 
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2.2.1  Perceptual  Benefits 

The  primary  benefit  to  be  gained  from  the  ability  to  display  information 
in  three  dimensions  is  the  opportunity  it  affords  to  relieve  the  viewer  of 
the  need  to  cognitively  translate  tabular  or  two  dimensional  graphic  data 
into  an  internal  representation  with  3D  characteristics.  Instead,  the  viewer 
can  perceive  more  rapidly  and  clearly  features  such  as  depth  and  relative 
location.  Perceptual  benefits  of  3D  are  of  four  general  types:  spatial 
features,  scaling  of  physical  attributes,  disambiguation,  and  abnormal 
sensorimotor  relationships.  These  are  discussed  in  greater  detail  below. 

The  world  in  which  we  live  is  a  three  dimensional  one,  composed  of 
objects  that  possess  mass,  occupy  space  and  obey  certain  invariant  rules  of 
physics  (e.g.,  more  than  one  object  cannot  occupy  the  same  space).  A  3D 
display  environment  can  provide  a  context  within  which  objects  and  their 
relationships  can  be  presented  and  interpreted  in  terms  of  familiar  spatial 
and  temporal  features.  For  example,  the  flight  profile  of  an  aircraft  or  the 
navigation  of  a  land  vehicle  might  be  easier  to  interpret  when  presented  in 
the  spatial  context  of  terrain  relief.  Invisible  but  definable  3D  areas, 
such  as  surface-to-air  missile  threat  envelopes,  can  be  displayed  as  volumes 
in  space,  providing  a  visualization  capability  not  possible  in  the  real  world 
but  obviously  valuable  when  presented  in  a  real  world  context.  More  creative 
uses  of  the  technology  are  available  to  applications  that  are  not  bound  by 
the  constraints  of  real  world  references  such  as  terrain.  Molecular 
modeling,  for  example,  is  one  field  benefiting  from  the  representation  of 
spatial  features  using  3D  displays. 

The  representation  of  objects  in  space  can  provide  a  scale  by  which 
estimates  of  physical  attributes  such  as  relative  size,  distance,  altitude, 
velocity  and  acceleration  can  be  derived.  For  example,  the  monocular  depth 
cues  available  in  the  imagery  from  a  navigation  sensor  can  provide  sufficient 
altitude  information  to  support  flight  control  under  certain  conditions.  A 
real-time,  interactive  3D  display  system  can  provide  additional  depth  cues 
(primarily  motion  parallax  cues)  to  generate  a  perception  of  speed  and 
acceleration. 

In  the  use  of  a  two  dimensional  display  system  for  graphics  and  time- 
sensitive  applications,  critical  data  and  images  can  accumulate  on  the 
display  to  the  point  of  distraction.  Air  traffic  control  is  one  example  of 
an  application  where  both  graphic  and  textual  information  are  necessary  in  a 
spatial  context.  The  addition  of  another  dimension  (i.e.,  depth)  affords  a 
way  of  decluttering  displayed  information  without  necessarily  reducing  it. 
Perceiving  graphic  and  textual  data  at  different  depths  allows  the  user  to 
focus  on  a  particular  depth  plane  and  filter  out  competing  visual  stimuli. 
In  interface  design  applications,  the  viewer  is  able  to  relate  spatial 
position  with  a  particular  function  such  as  the  location  of  a  pull  down  menu. 
Depth  in  this  case  need  not  be  associated  with  physical  distance,  but  can 
represent  a  conceptual  attribute  such  as  relative  importance  or  response 
priority. 

A  fourth  type  of  perceptual  benefit  of  3D  displays  is  the  unique 
opportunity  they  provide  for  simulating  abnormal  sensorimotor  relationships 
such  as  those  found  in  microgravity.  This  interactive  simulation  can  play  an 
important  role  in  training.  Evidence  suggests  that  information  about  visual 
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stimuli  can  be  retained  in  Short  Term  Memory  (STM)  in  a  manner  that  conveys 
the  physical  properties  of  the  elements  composing  the  stimuli.  In  addition, 
studies  of  mental  rotation  and  image  scanning  indicate  that  visual  STM  codes 
are  processed  much  like  actual  physically  present  objects.  STM  or  "working" 
memory  is  believed  to  be  the  site  of  ongoing,  conscious  activity.  If  STM 
visual  codes  strongly  resemble  the  physical  attributes  of  actual  3D 
structures,  then  performance  and  situation  awareness  might  be  enhanced  if  3D 
visual  stimuli  are  used  in  perceptual /motor  tasks.  When  the  normal 
relationship  between  proprioception  (activation  of  sensory  end  organs  in 
muscles,  etc.  that  sense  movement  in  the  body)  and  vision  is  distorted,  it 
has  been  found  that  adaptation  to  the  new  relationship  is  quicker  when  active 
(i.e.,  voluntary)  movement  is  allowed  than  when  the  subject  is  only  passively 
exposed  to  the  change.  In  this  vein,  3D  displays  would  be  beneficial  for 
simulating  the  visual  effects  produced  by  "abnormal"  sensorimotor 
relationships  in  pre-adaptation  trainers. 

2.2.2.  Cognitive  Benefits 

Although  the  major  advantage  to  the  user  of  displaying  information  in  3D 
is  the  shifting  of  some  tasks  from  cognitive  to  perceptual  processes,  there 
are  potential  cognitive  benefits  as  well.  Simply  stated,  cognitive  processes 
are  those  functions  performed  by  the  brain  to  transform,  manipulate,  store, 
retrieve,  analyze,  evaluate  and  utilize  information.  For  example,  the 
increased  perceptual  correspondence  that  3D  provides  of  display  imagery  with 
our  knowledge  of  the  external  environment  can  result  in  a  reduction  of  mental 
workload  and  an  enhanced  situation  awareness.  The  impact  of  3D  applications 
on  the  cognitive  processes  of  information  processing,  data  representation  in 
memory,  and  anticipation/expectancy  are  discussed  in  the  next  three  sections. 

2.2.2. 1  Information  Processing 

Although  the  term  "information  processing"  is  usually  very  broadly 
defined,  for  the  purpose  of  this  report  it  refers  to  those  transformations  of 
the  visual  signal  that  involve  more  "cognitive"  concepts,  such  as  memory. 

The  perception  of  a  third  dimension  provides  an  additional  dimension 
along  which  data  can  be  displayed  to  represent  how  they  are  related  to  each 
other.  A  very  natural  application  of  3D  would  be  the  display  of  terrain 
imagery  where  the  relative  distance  and  size  of  the  terrain  elements  (e.g., 
trees,  hills,  structures)  can  be  immediately  and  directly  assessed.  However, 
the  concept  of  relating  data  using  an  "analog"  representation  (see  Section 
2.Z.2.2  below)  has  also  been  adapted  to  develop  a  model  by  which  declarative 
representations  of  situations,  map  data  and  events  can  be  analyzed  and 
described.  This  process,  referred  to  as  Topological  Representation,  is  based 
on  a  formal  logical  theory  of  representation  for  situation  assessment  across 
two  and  three  dimensional  domains.  These  representations  can  form  a 
knowledge  base  or  be  logically  manipulated  by  symbolic  inference  algorithms. 
Topological  Representation  has  been  applied  to  route  planning  problems. 

Dynamic  networks  also  benefit  from  3D  representation.  Dynamic  networks 
can  be  pre-developed  for  an  application  (e.g.,  training,  demonstration),  or 
they  can  be  interactive  (e.g.,  man- in-the- loop  simulation).  Computer 
graphics  simulation  of  dynamic  three-dimensional  airspace  intercept  tactics 
and  geometry  have  been  used  as  part  of  the  Interceptor  Weapons  School  (IWS) 
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training  program.  According  to  Finegold,  Asch,  and  Flaugher  in  their 
technical  report.  Simulated  Three-dimensional  Graphics  Training  Display  for 
Air  Weapons  Controllers. 

"The  primary  training  problem  is  the  difficulty  of  effectively 
presenting  the  dynamic  geometry  involved  in  performing  intercepts.  For 
example,  instructors  cannot  easily  show  students  the  effects  of  the 
timing  of  a  turn  on  the  geometry  of  the  intercept,  nor  can  they  easily 
demonstrate  how  a  heading  correction  given  at  a  particular  time  will 
affect  the  intercept  two  or  three  minutes  later.  Most  importantly, 
students  cannot  be  shown  the  relationship  between  two-dimensional 
intercept  geometry  and  the  fact  that  aircraft  are  actually  flying  in  a 
three-dimensional  real  world.  The  two  dimensional  radar  display  does 
not  graphically  show  the  effect  of  altitude  separation  on  either 
intercept  geometry  or  flight  safety." 

Real-time,  interactive  dynamic  networks  are  used  to  simulate  the  real 
world  in  man-in-the-loop  training  paradigms  and  to  facilitate  remote  control 
of  vehicles,  weapons,  and  other  control  structures  (e.g.,  mechanical  "arms"). 
They  provide  a  context  from  which  the  spatial  and  temporal  features  of 
objects  and  events  can  be  easily  interpreted.  Real-time  dynamic  networks  can 
be  used  to  convey  interactive  events,  such  as  those  governing  the  cooperative 
operations  among  aircraft  in  a  flight  squadron  (tactical)  or  tanks  in  a 
platoon  (strategic). 

A  three-dimensional  data  structure  can  also  be  used  to  scale  abstract 
attributes  just  as  easily  as  the  physical  attributes  discussed  in  Section 
2.2.1.  Examples  of  abstract  attributes  might  include:  importance,  urgency, 
and  difficulty.  An  integrated  assessment  of  these  types  of  attributes  can  be 
represented  by  multidimensional  scaling  techniques  where  the  similarity  among 
pairs  of  items  are  measured  and  described  as  an  arrangement  of  the  items  in  a 
multidimensional  space.  Moreover,  the  dimensions  of  the  resulting  space  can 
be  used  to  make  inferences  about  the  basis  of  subjects'  similarity  ratings. 
Three-dimensional  scaling  of  abstract  attributes  would  be  beneficial  for 
applications  such  as  time  management,  task  structuring,  and  mission  planning. 

2. 2. 2. 2  Data  Representation  in  Memory 

Three-dimensional  displays  can  facilitate  information  retention  and 
retrieval.  Evidence  exists  for  analog  representation  of  information  in 
memory.  Analog  memory  codes  represent  information  that  seems  continuous  or 
has  properties  that  are  similar  to  actual  perception  (i.e.,  imagery).  For 
example,  even  when  patterns  are  represented  as  2D  drawings,  subjects  perceive 
the  patterns  as  3D  objects  and  can  mentally  "rotate"  them  in  different 
planes.  The  organization  of  the  elements  composing  the  analog  memory  code, 
however,  appears  to  be  an  important  factor  in  how  well  analog  information  Is 
retrieved  from  memory.  That  is,  people  are  better  at  remembering  meaningful, 
integrated  pictures  than  disorganized,  non-related  pictures. 

Generally  speaking,  pictures  are  more  easily  recognized  and  can  be  more 
easily  discriminated  from  one  another  than  words.  In  one  study,  subjects 
were  asked  to  repeatedly  try  to  recall  items  from  a  list.  When  the  items  had 
been  presented  as  a  list  of  words,  average  recall  performance  was  fairly 
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constant  after  the  first  hour.  When  the  items  were  presented  as  pictures, 
however,  recall  performance  continued  to  improve  for  about  four  days. 

A  number  of  studies  have  shown  that  people  have  a  vast  capacity  for 
storing  visual  details.  In  these  studies,  subjects  were  able  to  recognize 
600  previously  viewed  pictures  or  2560  previously  viewed  images  with  97  and 
90  percent  accuracy,  respectively.  This  capacity  is  far  greater  than  the 
ability  to  recognize  previously  viewed  words.  This  type  of  data  indicates 
that  a  specialized  code  for  visual  information  might  be  available  for  Long 
Term  Memory  (LTM). 

If  visual  information  in  LTM  is  encoded  to  preserve  the  physical 
attributes  of  the  visual  stimulus,  then  it  would  be  expected  that  3D  displays 
would  have  the  advantage  of  representing  these  physical  attributes  directly 
in  the  image  or  picture  presented  to  the  viewer.  Studies  have  shown  that 
when  meaningful,  integrated  pictures  or  images  are  used  to  represent 
information,  recall  for  the  location  and  identity  of  objects  is  very  good. 
Therefore,  3D  imagery  or  pictorial  formats  might  be  very  beneficial  to  the 
recognition  of  significant  objects  (e.g.,  targets,  threats,  way  points)  and 
to  the  display  of  spatially  distributed  resources  (e.g.,  stores,  weapons). 

There  is  also  a  variety  of  evidence  that  imagery  influences  memory 
performance  and  that  imagery  is  a  very  effective  mnemonic  strategy.  It 
appears  that  several  aspects  of  the  encoding  task,  including  context, 
generally  influence  the  retrieval  of  information.  Mnemonic  techniques 
provide  an  integrated  context  by  which  information  can  be  represented  in 
memory.  This  context  serves  as  a  set  of  potential  retrieval  cues.  A  well 
known  mnemonic  strategy,  referred  to  as  the  "method  of  loci,"  works  through 
spatial  imagery  and  is  very  effective.  These  types  of  strategies  are 
compatible  with  3D  representation  of  information  and  would  be  beneficial  for 
facilitating  memory,  especially  memory  for  spatial  information  (e.g., 
briefings,  intelligence  information). 

2 . 2 . 2 . 3  Ant i c i pat i on/Expectat i on 

There  is  evidence  that  expectancy  can  influence  perception  (e.g., 
recognition  of  degraded  pictures)  and  judgment  through  context  effects  and 
inference  cueing.  The  term  "Inference  cueing"  is  used  to  describe  the 
effects  of  preview  and  prediction.  That  Is,  displaying  a  preview  of  expected 
events,  actions  or  conditions  serves  to  bound  the  inferences  the  viewer  might 
make  with  respect  to  his/her  own  actions. 

For  example,  a  pictorial  representation  of  the  safest  route  for  a 
tactical  pilot  to  fly  during  a  particular  mission  phase  constrains  the  pilot 
to  certain  flight  control  actions.  The  objective  and  benefit  of  this 
"pathway  in  the  sky"  flight  path  Indicator  is  to  reduce  the  Information  load 
on  the  pilot  by  eliminating  costly  time  spent  In  gathering  Information 
traditionally  distributed  throughout  the  cockpit.  Disparate  data  are 
Integrated  Into  synthesized,  higher  level  display  variables  that  are  directly 
relevant  to  the  pilot's  decision  making  process. 

Representation  of  task  loading  is  another  application  for  3D  displays. 
In  situations  where  task  management  Is  Important,  tasks  that  need  to  be 
performed  can  be  displayed  within  a  3D  structure  where  depth  can  be  used  as 
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an  indicator  of  task  load,  task  priority,  etc.  Three-dimensional 
representation  of  Information  might  also  be  beneficial  for  predictive  flight 
control  displays  and  for  Indicating  potential  threat  conditions. 

More  abstract  data  Is  beginning  to  benefit  from  advances  In  the  new  area 
of  data  visualization  as  well.  Scientists  studying  overwhelming  volumes  of 
sensor  readings  from  interplanetary  satellites,  for  example,  may  be  able  to 
gain  important  new  insights  into  the  meaning  and  pattern  of  raw  data  given 
the  capability  of  displaying  it  graphically  in  three  dimensions. 


3.0  30  IN  C3I  APPLICATIONS 


The  primary  purpose  of  this  study  was  the  analysis  of  a  subset  of 
typical  C3I  battle  management  applications  and  the  subseguent  evaluation  of 
the  ways  in  which  three  dimensional  display  technology  could  aid  in  the 
accomplishment  of  mission  tasks.  One  part  of  this  analysis  included 
developing  an  understanding  of  possible  applications  which  rely  upon  data 
that  would,  theoretically,  lend  itself  to  a  three  dimensional  representation. 
A  discussion  of  this  analysis  is  provided  below  in  Section  3.1.  Another  part 
of  the  study  sought  to  identify  specific  unique  benefits  that  might  result 
from  the  3D  display  of  information.  The  results  derived  from  a  technology- 
independent  examination  of  information  utilized  in  the  course  of  battle 
management  activities  are  discussed  in  Section  3.2.  The  fundamental  reason 
for  conducting  this  study  was  to  provide  some  insights  into  which  battle 
management  tasks  should  see  an  improvement  in  performance  as  a  direct  result 
of  the  use  of  3D  display  capabilities.  Section  3.3  maps  the  benefits  of 
Section  3.2  to  specific  battle  management  applications  which  possess  the 
characteristics  described  in  Section  3.1. 

3.1  Candidate  Applications  and  Data  Types 

3.1.1  Inherently  30  Applications 

The  first  type  of  data  that  Is  considered  suitable  for  representation  in 
three  dimensions  is  data  with  inherent  3D  characteristics.  This  includes 
both  data  portraying  spatially  related  objects  such  as  terrain,  surface 
features  and  entities,  aircraft,  and  satellites  as  well  as  data  with  a 
spatial  component  like  sensor  coverage. 

3. 1.1.1  Object  Representation 

Three  classes  of  objects  are  candidates  for  representation.  Natural 
objects  like  terrain,  vegetation  and  water  can  be  used  to  provide  a 
compelling  and,  in  some  cases,  relatively  unchanging  spatial  environment  in 
which  other  activities  take  place,  while  naturally  occurring  events  like 
weather  conditions  and  natural  lighting  may  be  of  interest  as  well.  Man-made 
objects,  to  include  moving  vehicles  (land,  air,  sea,  space),  stationary 
features  like  towns  and  electronic  envelopes  such  as  radar  threat  domes  are 
imposed  on  a  natural  environment  and  represent  the  objects  of  greatest 
Interest  In  C3l  applications.  The  final  class  of  object  Is  text.  In 
addition  to  the  graphical  representation  of  an  object  or  entity,  text  Is 
necessary  for  providing  the  appropriate  descriptive  information  associated 
with  it,  information  that  cannot  be  represented  pictorial ly. 
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Data  representing  natural  feature  information  is  often  readily  available 
in  i  form  that  can  be  used  to  generate  3D  views.  One  problem  with 
representing  natural  scenes,  however,  is  that  of  scale.  While  the  capability 
exists  to  display  large  amounts  of  terrain  or  space,  the  relative  size  of 
human  scale  objects  within  that  space  can  become  miniscule.  Similarly,  as  a 
larger  area  of  terrain  is  viewed,  the  vertical  relief  diminishes.  This 
effect  can  be  offset  to  some  extent  by  changing  the  scale  of  objects  in  the 
scene,  exaggerating  the  vertical  dimension  or  replacing  objects  with 
representational  icons.  The  result,  however,  may  be  a  loss  of  accuracy  or 
confusion  as  to  the  actual  scale  of  objects  in  the  scene. 

Man-made  objects  representing  fixed  cultural  features  are  subject  to 
many  of  the  same  considerations  as  natural  features.  Mobile  objects  present 
unique  opportunities  and  challenges  in  a  3D  environment.  Many  of  the  issues 
of  concern  in  the  C3 1  arena  deal  with  the  interactions  between  moving 
objects.  Allied  and  enemy  ground  forces,  air  defense  activities,  locations 
of  targets  and  threats  all  provide  inputs  to  a  dynamically  changing  scenario. 
Motion  in  a  three  dimensional  space  provides  an  added  component  to  the 
viewer's  perception  of  the  situation.  In  addition  to  supplying  valuable 
monocular  motion  parallax  cues,  motion  in  three-space  can  allow  the  viewer  to 
more  accurately  perceive  acceleration  and  gauge  the  time  at  which  an  object 
will  reach  a  specific  point  in  space.  The  capabilities  afforded  by  movement 
in  30  can  be  very  important  for  both  the  mission  planner  objectively 
considering  the  interaction  of  a  number  of  objects  and  the  trainee 
subjectively  experiencing  simulated  encounters  in  close  air  support, 
intercept  or  reconnaissance  scenarios.  Data  describing  events  not  normally 
visible  to  the  human  eye  can  also  be  plotted  in  a  3D  environment.  A  3D 
volume  with  a  density  gradient  illustrating  the  effectiveness  of  radar 
coverage  for  a  particular  site  is  an  example  of  valuable  information  rendered 
graphically. 

The  display  of  text  is  a  vital  component  of  many  battle  management 
activities.  Detailed  descriptive  information  that  cannot  be  graphically 
illustrated  must  be  available  upon  command.  Some  problems  with  text, 
particularly  in  conjunction  with  a  graphic  display,  are  that  it  cannot  be 
displayed  over  other  text,  only  small  amounts  can  be  displayed  without 
cluttering  the  screen,  yet  it  must  be  displayed  large  enough  to  be  read. 
When  text  associated  with  several  objects  on  the  screen  must  be  displayed,  it 
is  difficult  to  ensure  that  the  correspondence  between  text  and  object  is 
obvious  while  maintaining  readability.  In  a  3D  environment,  some  measure  of 
relief  can  be  achieved  by  displaying  the  text  at  different  depths. 
Oecluttering  In  this  way  relieves  the  viewer  of  the  need  to  discriminate 
between  textual  messages  displayed  In  the  same  plane  and  facilitates  the 
association  of  text  with  objects  at  different  depths.  This  capability  is 
enhanced  by  the  judicious  use  of  shading  and  scaling  to  preserve  readability 
while  providing  additional  monocular  depth  cues. 

3. 1.1.2  Spatial  Viewpoints 

While  manipulation  of  a  2D  display  to  show  a  scene  from  various 
viewpoints  can  provide  the  viewer  with  a  sense  of  relative  locations  of 
objects,  similar  manipulation  of  the  viewpoint  in  a  three  dimensional  display 
system  allows  the  user  to  perceive  the  environment  from  a  selected  position 
in  three-space.  This  means  that  the  viewer,  from  a  single  selected  point  of 
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view,  can  determine  the  relative  distance  of  objects  within  the  field  of 
view.  With  magnification  (zoom)  and  pan  capabilities,  a  more  realistic 
assessment  of  the  situation  can  be  achieved  in  a  shorter  period  of  time, 
improving  the  performance  of  such  tasks  as  route  planning,  tactical 
simulation  and  training. 

3. 1.1. 3  Object  Manipulation  in  Spatial  Context 

With  the  ability  to  create  objects  in  a  three  dimensional  space  and  to 
view  them  from  any  desired  location  in  that  space,  the  manipulation  of  those 
objects  allows  the  user  to  create  dynamic  situations,  observe  object 
interactions  and  evaluate  the  consequences.  In  a  3D  environment,  rapid 
placement  and  orientation  of  objects  enables  the  operator  to  more  easily 
construct  a  scenario  with  the  desired  characteristics  and  improves  the  speed 
with  which  he  can  perform  this  task.  It  allows  for  easy  definition  of 
otherwise  difficult  data  such  as  flight  paths  or  deployment  patterns  and 
supports  rapid  testing  of  what-if  conjectures.  A  significant  technological 
limitation  in  this  area,  however,  is  the  lack  of  proven  input  devices  and 
user  interface  techniques  for  use  with  3D  displays.  Until  the  technology 
improves,  this  will  remain  one  of  the  primary  restrictions  to  full  user 
interaction  with  and  acceptance  of  3D  display  systems. 

3. 1.1.4  Volumes  and  Envelopes 

As  mentioned  briefly  above,  a  three  dimensional  viewing  space  can  be 
effectively  used  to  display  volumes  representing  radar  coverage  or  similar 
sensor  data.  Other  data  appropriate  for  representation  would  include  direct 
fire  weapon  effectiveness,  likelihood  of  detection,  regions  under  control  of 
specific  combat  units  and  anticipated  ECM  activity. 

In  mission  planning,  for  example,  detectability  is  a  composite  estimate 
of  the  range,  sensitivity,  and  scope  of  the  enemy's  detection  system,  e.g., 
radar,  and  the  effectiveness  of  the  detection-evasion  strategy  of  the  weapon 
system  platform.  Terrain  might  be  a  major  consideration  in  determining  the 
scope  of  the  enemy's  detection  system.  For  example,  radar  shadows  might 
exist  on  the  leeward  side  of  mountains,  thereby  reducing  the  scope  of 
detection.  From  an  analysis  of  all  of  these  factors,  one  could  generate 
volumetric  representations  of  the  probability  of  detection  by  the  enemy's 
radar. 

Placement  of  the  radar  unit,  its  power,  and  the  characteristics  of 
adjacent  terrain  all  affect  the  shape  of  the  detection  canopy  surrounding 
each  radar  installation.  Because  of  terrain  shadowing,  the  probability 
canopies  will  not  have  symmetrical  spherical  fronts.  Thus  the  intersections 
of  iso-detection  probability  fronts  from  adjacent  radar  installations  may  not 
describe  circles,  and  the  volume  described  by  their  union  may  be  irregular. 

Similar  considerations  apply  to  the  enemy's  threat  capability.  The 
placement,  range,  accuracy,  and  scope  of  the  enemy's  weapon  systems  can  be 
used  to  develop  three-dimensional  potential  threat  volumes  along  alternative 
flight  paths  to  the  target.  The  intersections  of  threat-potential  fronts  can 
be  used  to  predict  loci  of  maximum  permissible  threat  consistent  with  mission 
objectives.  These  loci  will  not  be  planar,  so  their  depiction  on  a  2D 
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display  may  be  difficult  to  interpret.  Interpretation  should  be  greatly 
facilitated  by  the  3D  presentation  of  the  iso-threat  loci. 

Ideally,  a  3D  display  might  present  a  three-dimensional  rendition  of  the 
terrain  along  alternate  flight  paths,  as  well  as  ghosted  views  of  both  the 
detection  fronts  (at  a  predetermined  detection  probability)  and  the  threat 
fronts  (likewise  at  a  predetermined  tolerable  threat  potential.)  It  may  be 
necessary  to  encode  detection  and  threat  fronts  differentially,  e.g.,  by 
color,  so  that  the  viewer  can  assess  locations  in  three-space  that  present 
the  best  compromise.  For  example,  there  may  be  altitudes  at  particular  map 
coordinates  where  the  threat  potential  exceeds  the  tolerance  threshold,  but 
the  probability  of  detection  at  this  location  is  very  low. 

To  maximize  the  benefit  from  the  displayed  information,  the  viewer  might 
synthesize  a  perceptual  volume  surrounding  the  aircraft  which  shows  the 
location  of  the  predetermined  threshold  values  of  the  threat  and  detection 
variables.  The  viewer  might  also  have  the  ability  to  select  other  values  of 
each  variable  so  that  he  can  assess  the  joint  probability  of  detection  and 
threat  at  any  displayed  location.  Decisions  can  then  be  based  on  whether  the 
proposed  flight  path  pierces  the  detection  and  threat  fronts,  where  piercing 
the  front  means  exceeding  the  present  value. 

3.1.2  Other  3D  Representations 

In  addition  to  the  data  described  above  which  possesses  inherent  spatial 
characteristics,  certain  forms  of  data  that  do  not  possess  such  obvious 
spatial  features  can  benefit  from  a  three  dimensional  representation  as  well. 
The  growing  field  of  data  visualization  provides  an  indication  of  the 
widespread  interest  in  finding  new  and  flexible  means  for  displaying  and 
examining  information.  Applications  for  data  visualization  tools  range  from 
debugging  large  amounts  of  unprocessed  data  and  determining  the  most 
promising  directions  for  research  to  extracting  and  effectively  presenting 
previously  undetected  relationships  among  the  data.  Three  dimensional 
displays  provide  the  user  with  an  additional  axis  along  which  information  can 
be  plotted.  Some  basic  examples  of  applications  in  the  C3l  environment  are 
discussed  below.  Other  possibilities  for  presenting  information  in  a  three 
dimensional  environment  are  primarily  restricted  by  the  imaginations  of  the 
researcher,  the  software  developer  and  the  end-user.  For  professionals 
trained  and  experienced  in  preparing  data  for  display  in  two  dimensions, 
making  effective  use  of  spatial  display  capabilities  can  present  a 
substantial  creative  and  technical  challenge. 

3. 1.2.1  Networks 

A  form  of  data  representation  that  can  be  enhanced  by  3D  is  the  network. 
The  addition  of  depth  allows  for  the  display  of  more  complex  systems  and 
relationships  than  a  2D  rendition.  Some  examples  include  communi cations  nets 
to  Illustrate  who  communicates  with  whom  as  a  function  of  mission  scenario 
and  time,  and  dynamic  networks  for  training  purposes  (as  described  in  Section 
2.2.2. 1)  in  which  the  network  shows  the  relationships  between  complex  sets  of 
variables,  such  as  the  effect  of  the  timing  of  a  turn  in  a  particular  fighter 
aircraft  on  the  geometry  of  the  target  intercept.  Another  form  of  network 
displays  are  management  charts  showing  organizational  structure  and  PERT 
charts  for  purposes  of  tactical  and  logistics  planning  and  assessment. 


3. 1.2. 2  User  Interface 


The  field  of  man-machine  interfaces  has  several  possible  applications 
areas  for  3D  technology.  Multiple  spatial  displays  can  exist  at  different 
depth  planes,  in  contrast  to  the  current  technique  of  multiple  windows  as  an 
analogy  to  a  desktop.  Three  dimensional  menuing  systems  could  facilitate  the 
selection  of  interrelated  options.  A  spatial  hypermedia  facility  would  allow 
for  rapid  progression  through  linked  spatial  displays  along  some  line  of 
interest  to  the  user.  The  implementation  issues  associated  with  these 
concepts  are  essentially  extensions  of  existing  display  management  and 
information  retrieval  issues  encountered  in  developing  2D  systems. 

3.2  Tentative  Benefits  to  Battle  Management 

In  examining  the  unique  benefits  that  a  three  dimensional  display 
capability  offers  battle  management  applications,  it  was  necessary  first  to 
consider  the  capabilities  offered  by  more  conventional,  perspective  2D 
technology  or  2  1/2  D.  One  conclusion  reached  during  this  process  was  that 
perspective  2D  capabilities  are  often  not  fully  exploited.  As  hardware  costs 
drop  and  processing  speed  increases,  the  display  capabilities  of  graphics 
workstations  continue  to  expand  tremendously.  With  the  ability  to  manipulate 
scenes  and  move  within  the  2  1/2  D  environment,  perspective  displays  can 
rival  some  features  of  3D  displays  without  imposing  the  computational  demands 
required  for  spatial  image  generation.  To  some  extent,  this  increasing 
availability  of  good  perspective  graphics  has  probably  had  the  effect  of 
limiting  the  aggressive  development  of  3D  technology.  Until  technological 
advances  and  user  experience  allow  the  appropriate  applications  areas  for  3D 
displays  to  be  more  clearly  defined,  perspective  2D  technology  applied  with 
creativity  and  innovation  can  and  will  continue  to  meet  the  needs  of  many 
appl ications. 

Three  tentative  benefits  resulting  from  the  use  of  3D  displays  in  battle 
management  applications  were  identified  during  the  course  of  the  study. 
These  benefits  were  derived  from  a  technology-independent  examination  of 
information  utilized  in  the  course  of  battle  management  activities.  They 
are: 


-  decluttering 

-  the  display  of  irregular  volumes 

-  the  display  of  three  dimensional  scenes  over  time 

Artificial  limitations  are  imposed  on  the  presentation  of  information 
via  computer  monitors  as  a  result  of  the  size  of  the  screen,  the  resolution 
of  the  image  and  the  issues  raised  by  displaying  overlapping  objects  and 
text.  The  exponential  increase  in  display  space  provided  by  the  addition  of 
a  third  dimension  offers  a  means  of  resolving  these  display  problems  that 
restrict  operational  capabilities.  Information  presented  at  different  depth 
planes  declutters  the  image  perceived  by  the  viewer  by  reducing  the  data 
presented  in  any  one  plane  and  allowing  him  to  selectively  focus  on  it 
without  distraction. 

In  addition  to  providing  a  way  to  separate  individual  objects  spatially, 
the  3D  display  of  information  enables  the  viewer  to  perceive  three 
dimensional  objects  as  possessing  depth  themselves.  The  form  of  regular. 
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geometrically  shaped  volumes  can  be  inferred  from  perspective 
representations.  Irregularly  shaped  volumes,  however,  such  as  those 
depicting  realistic  electronic  sensitivity  or  projection  envelopes,  cannot  be 
adequately  perceived  by  viewing  the  image  on  a  two  dimensional  surface. 
Similarly,  the  intersection  of  multiple  volumes  is  most  likely  not  planar, 
and  depiction  as  a  planar  form  is  both  misleading  and  inaccurate.  A  major 
benefit  of  30  displays  is  their  capability  to  provide,  in  a  form  detectable 
to  the  viewer,  information  as  volumes  as  opposed  to  information  as  planar 
areas.  This  increased  quantity  of  information,  and  the  complex  forms  it  can 
take  when  volumes  intersect,  can  only  be  perceived  in  a  three  dimensional 
display  environment. 

The  final  advantage  of  30  displays  involves  the  movement  of  the  objects 
and  volumes  described  above.  This  essentially  allows  the  viewer  to  perceive 
a  fourth  dimension,  that  of  time.  A  three  dimensional  display  space  not  only 
allows  for  the  display  of  object  in  space,  but  for  their  movement  in  space  as 
well.  The  ability  to  perceive  relative  motion  in  three  dimensions  results  in 
a  more  rapid  and  accurate  estimation  of  speed,  velocity  and  time  at  which  an 
object  will  reach  a  certain  point  in  space  or  intercept  another  object.  The 
display  of  this  information  on  a  two  dimensional  screen  is  very  difficult  to 
accomplish. 

3.3  Specific  Battle  Management  Applications 

In  discussions  regarding  the  C3I  scope  of  the  study,  RADC  and  the  team 
agreed  that  attention  should  be  focused  primarily  on  the  functions  of 
situation  assessment  and  operational  mission  planning  in  an  immediate  post¬ 
hostilities  timeframe.  This  section  identifies  aspects  of  these  functions 
that  would  seem  to  benefit  the  most  from  3D  advantages  described  above. 

3.3.1  Situation  Assessment 

Situation  assessment  as  a  function  of  C3I  includes  maintaining 
information  on  the  status  of  a  variety  of  factors:  friendly  and  allied 
forces  (air,  ground,  naval);  enemy  forces  (same  as  for  friendly  and  allied 
plus  threat  assessment);  the  battlefield  (combat  assessment);  and  other 
dynamics  of  the  battlefield  such  as  logistics,  environmental  factors,  non- 
combatants  and  obstacles.  Most  of  these  situation  assessment  functions 
involve  the  positions,  conditions,  activities  and  capabilities  of  man-made 
and  natural  entities  within  a  very  well  defined  physical  space  and,  hence, 
lend  themselves  to  representation  in  a  3D  display  environment.  The  use  of  3D 
for  a  static  display  of  forces  and  related  information  could  provide  benefits 
in  terms  of  decluttering  the  displayed  image,  but  in  most  cases  represents  an 
increased  overhead  cost  with  no  substantial  improvement  over  current 
practices.  Greater  benefits  are  realized  by  the  more  active  use  of  the 
display  medium. 

Two  fundamental  opportunities  exist  to  use  the  3D  display  environment  to 
exhibit  the  motion  of  forces.  The  first  of  these  illustrates  the  historical 
movement  of  military  and  natural  forces  within  the  geographic  area  of 
Interest.  Much  of  the  information  maintained  as  part  of  the  situation 
assessment  function  Is  fixed  (though  not  always  known)  at  any  given  time.  A 
display  of  the  movement  of  entitles  that  led  to  the  current  situation  would 
indicate  the  direction  and  force  of  tactical  thrusts.  Presented  In  3D,  the 
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display  could  provide  insights  into  the  effect  of  terrain  on  movement.  In 
illustrating  movement  of  air  and  naval  forces,  the  3D  display  could  be 
particularly  effective  in  imparting  a  sense  of  distance  between  forces  moving 
in  three  dimensional  space.  The  second  use  of  motion  in  situation  assessment 
involves  monitoring  indications  and  warning  factors  by  projecting  force 
movement  into  the  relatively  near  future  to  determine  possible  consequences 
of  current  activity.  Closely  coupled  with  this  application  is  the  monitoring 
of  environmental  and  other  dynamic  factors  of  the  battlefield  to  project 
their  effect  on  the  current  and  anticipated  states  of  the  battlefield. 

More  appropriate  to  situation  assessment  activities  is  the  ability  of  3D 
to  provide  a  more  accurate  view  of  spatial  volumes  and  their  intersections. 
The  status  information  of  all  military  forces  includes  details  of  electronic 
warfare  characteristics  such  as  coverage,  effective  radiated  power, 
collection  radius  of  sensors  and  SIGINT  systems,  and  vulnerability  to 
countermeasures.  Other  data  that  can  be  represented  by  volumes  or  envelopes 
includes  weather  and  contamination  data.  The  usefulness  of  3D  in  viewing 
data  that  is  inherently  spatial  in  nature  is  that  it  provides  a  more  complete 
and  realistic  perception  of  the  relative  sizes  and  positions  of  the  various 
entities,  particularly  as  they  interact  over  time. 

In  summary,  the  applications  associated  with  C3I  situation  assessment 
that  would  appear  to  benefit  the  most  from  the  use  of  3D  displays  are  those 
which  support  a  combination  of  dynamic  display  techniques,  i.e.,  movement 
over  time  and  the  representation  of  data  by  volumes.  The  display  of  the 
battlefield  situation  as  perceived  over  time,  illustrating  appropriate  data 
volumetrical ly,  would  take  maximum  advantage  of  the  anticipated  benefits  of 
3D  displays.  Perhaps  less  beneficial  but  more  widely  applicable  would  be  the 
use  of  3D  to  indicate  necessary  status  information.  The  ability  to  overlay 
on  the  battlefield  area  map  status  information,  be  it  inherently  spatial 
information  such  as  line-of-sight  for  indirect  fire  weapons  or  more  abstract 
information  like  mission  effectiveness  and  damage  assessment,  is  a  useful  and 
identifiable  capability  improved  by  3D  displays.  More  advanced  data 
visualization  techniques  may  offer  other  possibilities  as  well. 

3.3.2  Operational  Mission  Planning 

Operational  mission  planning  seeks  to  devise  and  accomplish  tactical 
missions  in  support  of  the  more  strategic  goals  determined  during  the  battle 
planning  stage.  In  contrast  to  situation  assessment,  which  strives  to 
determine  the  state  of  many  aspects  of  the  battlefield  at  any  given  time, 
mission  planning  attempts  to  determine  the  likely  results  of  many  possible 
and  interrelated  courses  of  action.  Facets  of  operational  mission  planning 
include  target  system  development  in  which  targets  are  defined  and 
categorized,  targeteering  for  identifying  the  target,  weapon  allocation  for 
the  mission,  weapon  application  and  evaluation,  FRAG  order  production  and 
plan  analysis  comparing  predicted  effectiveness  with  its  cost.  Development 
and  evaluation  of  plans  requires  the  ability  to  simulate  their  execution  and 
examine  their  performance  at  many  stages.  The  ability  to  change  aspects  of 
the  modeled  plan  in  response  to  perceived  threats  provides  the  user  with  a 
"what-if"  flexibility  for  developing  the  most  effective  plan.  The  activities 
and  requirements  of  the  mission  planning  function  map  very  well  to  the  data 
types  and  anticipated  benefits  described  above. 
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The  most  obvious  use  of  30  technology  in  operational  mission  planning  is 
the  visual  modeling  of  mission  execution  from  a  mission  planner's  viewpoint. 
3D  can  provide  the  spatial  image  of  the  appropriate  area  of  interest. 
Conventional  graphics  handling  routines  supply  the  algorithms  necessary  in 
most  cases  for  manipulation  of  the  spatial  image  (i.e.,  rotation, 
magnification)  while  supporting  manipulation  of  objects  within  the  image 
(redirection  of  forces,  etc.).  Because  viewing  the  execution  of  the  proposed 
mission  over  some  period  of  time  is  critical  to  the  planner,  the  motion  of 
objects  and  their  interaction  with  volumes  in  3D  can  provide  a  real  benefit. 

The  simulation  of  mission  execution  can  be  viewed  from  another 
perspective  as  well,  that  of  the  mission  pilot.  Targets  may  be  fixed  or 
mobile,  point  targets  or  an  area.  Battlefield  conditions  will  vary  depending 
on  time  of  day,  time  of  year,  weather  and  the  combat  situation.  To  improve 
target  identification  and  provide  some  advanced  insights  into  conditions 
affecting  the  mission,  a  pilot's  eye  view  of  the  mission  can  be  displayed  and 
can  be  used  to  explore  the  effects  of  various  tactical  moves  and  responses  in 
the  execution  of  the  mission.  Many  of  the  options  available  in  the  mission 
planner's  view  of  the  scene  would  be  appropriate  to  the  pilot's  view  as  well. 
Being  able  to  'see'  the  radar  detection  envelope  for  a  specific  probability 
of  detection  would  be  very  useful  as  the  aircraft  moves  around  it,  for 
example.  The  pilot's  view  of  the  scene  is,  however,  on  a  significantly 
different  scale  from  the  mission  planner's.  Objects  are  closer  and  distances 
between  them  are  more  clearly  perceived.  Where  the  planner  views  the 
aircraft  and  threat  envelope  from  a  distance,  the  pilot  sees  the  surface  of 
the  volume  closing  in  as  he  approaches  it.  Because  of  these  features,  this 
application  is  a  particularly  strong  candidate  for  enhancement  by  3D  display 
technology. 


4.0  3D  DISPLAY  TECHNIQUES 

This  section  examines  a  variety  of  existing  three  dimensional  display 
techniques  for  their  utility  in  the  C3I  arena  of  applications.  Because  each 
technology  has  different  features  and  drawbacks  that  determine  its 
applicability  to  particular  tasks,  technology-specific  viewing  factors  are 
discussed  as  they  apply  to  each  technique.  These  factors  may  include  the 
ease  of  fusion  of  stereo  images,  binocular  rivalry,  flicker,  pseudoscopic 
zones,  and  the  presumed  or  demonstrated  propensity  to  induce  visual  fatigue. 

In  addition  to  parameters  associated  with  the  display  techniques,  the 
discussions  will  include  an  examination  of  several  features  considered 
important  or  desirable  to  C3I  battle  management  applications.  These  factors 
Include  image  resolution,  color,  real-time  display  capability,  full  parallax, 
multi-viewer  support  and  the  requirement  for  user-worn  aids.  Image 
resolution  addresses  the  degree  of  detail  that  can  be  viewed  on  the  display. 
Depending  upon  the  technology  used,  the  availability  of  adequate  resolution 
and  the  prospect  for  Increasing  resolution  may  be  issues  of  concern.  Color 
Is  a  highly  desirable  feature  because  It  provides  both  an  added  realism  to 
graphic  images  and  an  additional  dimension  for  indicating  information.  A 
real-time  display  capability  Is  essential  for  C3I  applications.  While  less 
dynamic  display  requirements  may  be  sufficient  for  some  disciplines,  battle 
management  Is  characterized  by  its  need  for  Immediate  and  projected  status 
information  in  a  highly  volatile  environment.  Full  parallax  means  that  as 
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the  viewer  moves  his  head  to  the  right,  for  example,  he  is  able  to  see  more 
of  the  right  side  of  a  viewed  object.  The  effect  is  the  same  as  rotating  the 
object  to  the  left.  Vi ewer- induced  head  motion  parallax  allows  the  viewer  to 
perceive  additional  information  in  a  very  natural  manner.  Display  systems 
that  lack  this  feature  can  provide  confusing  viewing  cues  to  the  user  by 
making  it  appear  that  a  viewed  object  rotates  to  the  right  as  the  viewer 
moves  his  head  to  the  right.  The  ability  for  a  displayed  image  to  be 
perceived  clearly  by  multiple  viewers  is  important  in  battle  management 
situations  where  the  displayed  image  provides  information  for  evaluation  and 
discussion  by  several  people.  Two  components  comprise  this  requirement:  the 
need  for  the  generated  image  to  be  viewable  from  different  positions  relative 
to  the  display  and  the  need  for  the  perception  of  the  image  to  be  relatively 
independent  of  viewer  variability.  Finally,  for  the  applications  within  the 
scope  of  this  report,  the  need  for  the  viewer  to  wear  glasses  or  other  aids 
is  generally  considered  to  be  an  undesirable  restriction. 

4.1  Stereoscopic  Techniques 

Stereoscopic  techniques  involve  the  presentation  of  two  slightly  offset 
images  of  a  view  to  each  of  the  viewer's  eyes.  The  offset  duplicates  the 
different  perspective  views  of  the  two  eyes,  providing  a  perception  of 
binocular  depth.  The  problem  associated  with  providing  stereoscopic  views  is 
the  manner  in  which  the  visual  access  to  each  eye  is  restricted  to  the 
appropriate  image.  Section  4.1.1  reviews  techniques  in  which  the  images  are 
separated  by  the  viewer,  often  through  the  use  of  viewer-worn  glasses. 
Section  4.1.2  examines  several  techniques  which  separate  the  image  at  the 
display  source.  Many  of  these  systems  are  autostereoscopic,  meaning  they  do 
not  require  user-worn  aids  for  the  perception  of  depth  via  stereopsis. 

4.1.1  Viewer  Separation  of  Images 

4. 1.1.1  Chromostereoscopic 

All  types  of  chromostereoscopic  (or  color  stereoscopic)  display  systems 
rely  on  chromatic  aberration  to  provide  the  impression  of  depth.  Most  human 
eyes  have  a  small  amount  of  chromatic  aberration  caused  by  the  displacement 
of  the  visual  axis  (the  line  of  sight)  from  the  optic  axis  (the  line  normal 
to  the  cornea  and  lens  of  the  eye).  Because  the  magnitude  of  aberration 
varies  with  wavelength,  light  of  some  colors  is  refracted  more  than  light  of 
other  colors.  For  example,  when  two  juxtaposed  points  of  light,  one  red  and 
one  blue,  are  viewed  with  the  left  eye,  they  will  be  seen  as  separated  from 
each  other;  the  red  light  will  appear  to  be  to  the  left  of  the  blue  light. 
When  they  are  viewed  with  the  right  eye,  the  red  light  will  be  refracted  to 
the  right  of  the  blue  light.  As  a  result,  when  the  lights  are  viewed  with 
both  eyes,  they  will  be  imaged  with  binocular  disparity.  This  disparity  will 
make  the  red  light  appear  closer  to  the  observer  than  the  blue  light,  even 
though  they  are  physically  the  same  distance  from  the  viewer.  Display 
systems  based  on  chromostereopsis  have  the  advantage  of  requiring  a  single 
display  surface,  such  as  a  CRT  or  dichroic  LCD,  which  can  be  viewed  by 
several  viewers  simultaneously.  This  type  of  stereo  display,  however, 
exhibits  a  number  of  fundamental  limitations. 

Chromostereoscopic  displays  suffer  the  disadvantage  of  requiring  stimuli 
that  are  to  appear  in  different  planes  to  be  appropriately  colored.  Where 
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the  three-dimensional  scene  to  be  imaged  is  composed  of  chromatic  objects, 
chromostereopsis  may  produce  depth  cues  that  conflict  with  real  depth  cues 
like  perspective  and  parallax.  Furthermore,  the  angle  between  the  visual  and 
optic  axes  varies  widely  among  people,  leading  to  a  wide  range  of  perceived 
depth  planes  across  observers  for  the  same  physical  stimulus.  Some  observers 
even  see  the  chromatic  stimuli  in  the  opposite  depth  relationship,  i.e.  blue 
closer  than  red. 

There  are  two  particular  forms  of  color  stereoscopic  systems  that  merit 
identification.  Stenopeic  displays  enhance  depth  resolution  by  artificially 
displacing  the  observer's  visual  axes.  This  displacement  is  produced  by  a 
pair  of  stenopeic  glasses  that  force  the  observer  to  view  the  display  surface 
through  a  pair  of  pinholes  that  have  been  positioned  in  otherwise  opaque 
masks  so  as  to  separate  the  two  visual  axes.  This  increases  the  separation 
of  the  optic  and  visual  axes,  thereby  enhancing  the  chromostereoscopic 
effect.  The  glasses  dramatically  reduce  the  observer's  field  of  view  and 
promote  rapid  visual  fatigue. 

The  second  form  of  chromostereoscopic  display  is  the  prismatic  type, 
which  has  somewhat  fewer  disadvantages  than  the  stenopeic  type  although  it 
still  requires  the  viewer  to  wear  spectacles.  In  this  implementation,  the 
effect  is  enhanced  by  prisms  worn  in  front  of  the  eyes.  The  prisms  increase 
dispersion  of  the  incident  chromatic  rays,  resulting  in  increased  binocular 
disparity  and,  therefore,  perceived  depth.  By  using  adjustable  prisms,  such 
as  Risley  prisms,  it  is  possible  to  vary  the  depth  resolution  and  to 
standardize  the  perceived  depth  across  observers.  However,  even  the  simple 
prism  spectacles  are  cumbersome  and  presumably  would  promote  rapid  visual 
fatigue  due  to  the  phorias  they  induce. 

In  terms  of  applicability  to  C3I,  chromostereoscopic  systems  suffer  from 
their  inability  to  support  full  color  and  the  need  in  some  instances  for  the 
viewer  to  wear  cumbersome  glasses.  In  addition,  although  the  display  surface 
can  be  viewed  by  multiple  viewers,  the  technique  is  so  sensitive  to 
differences  in  observer-specific  vision  characteristics  as  to  be  considered 
unreliable  at  best.  The  technique  itself  imposes  rJ  constraints  on 
resolution,  but  its  use  in  the  real  time  depiction  ^  .,cion  would  require  a 
constant  and  distracting  shift  in  the  colors  oi  uividual  objects  as  their 
positions  changed.  Full  parallax  is  not  preserved  in  this  system.  In 
summary,  the  technique  is  neither  dynamic  nor  robust  enough  for  C3I 
appl ications. 

4. 1.1.2  Pulfrich  Phenomenon 

When  the  luminance  of  binocular  images  is  different  in  the  two  eyes,  the 
processing  time  for  each  image  differs  by  several  milliseconds.  With  static 
images  this  has  no  perceptual  effect,  but  with  dynamic  images  it  induces  an 
illusion  of  curvature  in  the  pathways  of  moving  objects.  An  object  moving  in 
a  straight  line  across  the  observer's  field  of  view  at  right  angles  to  his 
line  of  sight  appears  to  move  In  an  arc.  When  the  image  in  the  left  eye  is 
of  lower  luminance  than  that  in  the  right,  objects  moving  to  the  right  appear 
to  move  through  an  arc  that  is  convex  to  the  observer.  Thus,  an  object 
moving  back  and  forth,  such  as  a  pendulum,  would  appear  to  be  describing  an 
elliptical  or  even  circular  path. 
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Stereo  displays  based  upon  the  Pulfrich  phenomenon  have  several 
disadvantages  that  make  them  impractical  for  most  applications.  The  most 
severe  limitation  is  that  they  only  produce  an  impression  of  motion-in-depth 
rather  than  an  impression  of  depth,  per  se.  In  addition,  these  displays 
require  that  the  direction  of  fronto-parallel  motion  in  the  image  be 
controlled  in  order  to  produce  motion  in  depth  of  the  intended  polarity. 
That  is,  depending  upon  which  eye  is  viewing  the  dimmer  image,  motion  from 
left  to  right  will  always  follow  a  convex  (or  concave)  path  while  motion  in 
the  other  direction  will  always  be  the  opposite.  This  obviously  reduces  the 
operational  value  of  the  technique  in  applications  where  motion  may  occur  in 
many  directions  and  where  depth  perception  is  desirable  even  in  the  absence 
of  motion.  Pulfrich-based  systems  also  require  that  one  of  the  binocular 
images  be  of  a  lower  luminance  than  the  other,  which  is  most  easily 
accomplished  by  having  an  observer  wear  a  neutral  density  filter  (one  that 
reduces  the  intensity  of  the  image  without  changing  it<;  color)  in  front  of 
one  eye.  This  required  difference  in  luminance  tends  to  produce  binocular 
rivalry  and  visual  fatigue,  though  color  and  resolution  do  not  themselves 
suffer  as  result.  User-worn  aids  are  required,  often  in  the  form  of  passive 
eyeglasses.  Multiple  viewers  can  perceive  the  sense  of  depth  from  a  single 
display.  Like  the  chromostereoscopic  systems,  the  fundamental  limitations 
imposed  on  control  of  the  image  render  this  technique  inappropriate  for 
battle  management  applications. 

4. 1.1.3  Color  Anaglyph 

Color  anaglyphs  separate  pairs  by  color  subtraction.  Two  perspective 
images  of  the  same  three  dimensional  scene  are  presented  simultaneously  on 
the  display  screen.  The  disparate  regions  of  one  image  are  encoded  in  short 
wavelength  light,  and  those  of  the  other  image  in  long  wavelength  light. 
Where  there  is  no  image  disparity,  the  stereo  pair  can  be  encoded  in  full 
color.  The  viewer  wears  spectacles  comprising  one  short-wavelength  lens  and 
one  long-wavelength  lens.  Each  lens  selectively  transmits  significantly  more 
of  one  of  the  stereo  images,  thereby  effecting  monocular- image  separation. 
However,  because  each  spectacle  lens  must  transmit  light  from  the  entire 
spectrum  in  order  for  the  viewer  to  perceive  full-color  stereoscopic  images, 
the  optical  density  of  each  lens  cannot  be  very  great.  That  is,  each  lens 
must  not  be  too  dark  or  transmit  too  narrow  a  range  of  the  visible  spectrum. 
This  produces  annoying  cross-talk  between  the  left  and  right  eye-channels, 
which  is  perceived  as  ghost  images. 

This  is  an  old  technology  that  lends  itself  to  both  static  and  dynamic 
displays.  It  has  been  used  to  produce  3-D  comic  books  and  3-D  motion 
pictures.  A  3-D  television  system  has  also  been  based  on  color  anaglyphs. 
In  this  embodiment,  the  amount  of  disparity  is  greatly  reduced,  relative  to 
that  used  in  cinema  and  hard  copy.  This  produces  a  full-color  image  that  can 
be  viewed  with  or  without  the  color-anaglyphic  spectacles.  When  the 
spectacles  are  not  worn,  the  color  TV  image  has  some  blurry  color  fringes 
adjacent  to  three-dimensional  objects,  but  because  the  amount  of  disparity 
has  been  kept  to  a  minimum,  the  color  fringes  are  not  a  severe  detriment  to 
image  quality.  Also,  the  width  of  the  fringes  is  greatest  in  objects  that 
have  the  greatest  depth,  but  due  to  the  visual  system's  limited  depth  of 
field,  these  objects  are  normally  perceived  as  blurry.  When  the  spectacles 
are  worn,  the  viewer  perceives  a  stereoscopic  Image  of  the  scene  presented  on 
the  screen  because  of  the  binocular  disparity  encoded  in  the  color  fringes. 
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Image  separation  by  the  color  anaglyph  method  produces  some  undesirable 
phenomena.  Because  it  is  impossible  to  balance  the  transmission  of  the  long- 
and  short-wavelength  lenses  for  all  combinations  of  viewer  and  display,  there 
is  frequently  a  difference  in  effective  optical  density  between  the  left-  and 
right-eye  lenses.  This  tends  to  produce  both  binocular  rivalry  and  the 
Pulfrich  phenomenon,  as  well  as  visual  fatigue. 

The  blurring  of  the  image  reduces  the  functional  resolution  of  the 
image.  Color  and  real-time  processing  are  both  largely  supported,  though 
some  flexibility  in  the  use  of  color  is  necessarily  sacrificed  to  produce 
stereo  images.  Multiple  viewers  can  share  a  display,  though  the  system  is 
somewhat  sensitive  to  viewer  variability.  Glasses  are  required  for  3D  use, 
but  the  viewer  can  operate  in  2D  without  them.  Full  parallax  is  not 
maintained.  In  spite  of  its  use  in  feature  length  motion  pictures,  3D  based 
on  color  anaglyph  is  only  marginally  effective  and  too  visually  fatiguing  for 
long  term  use  in  battle  management. 

4. 1.1.4  Polarization  Stereogram 

More  a  technique  for  achieving  binocular  image  separation  than  a  display 
type  per  se,  the  polarization  stereogram  is  possibly  the  most  common.  In 
contrast  to  chromostereoscopic  and  Pulfrich  phenomenon-based  techniques  which 
rely  on  side  effects  of  human  vision  (and  consequently  suffer  from  its 
variability),  the  polarization  stereogram  presents  two  different  perspective 
views  to  the  viewer  in  much  the  same  manner  as  the  color  anaglyph  method. 
However,  the  use  of  polarized  rather  than  colored  filters  provides  for  more 
effective  image  separation  while  not  affecting  the  color  of  the  scene.  Each 
eye  views  the  display  surface  through  a  polarizer  that  is  orthogonal  to  its 
partner.  The  polarizers  separate  the  left-  and  right-eye  images  that  are 
presented  on  the  display  screen  simultaneously  or  alternately  at  a  rate  above 
flicker  fusion.  Two  forms  of  polarization  can  be  used  for  this  display  type: 
linear  polarization  and  circular  polarization. 

In  most  applications,  the  viewer  wears  spectacles  that  incorporate 
polarizers  that  must  remain  orthogonal  to  their  companion  polarizers  at  the 
display  surface.  Cross-talk  occurs  for  several  reasons.  Crossed  polarizers 
are  not  perfect  attenuators:  they  exhibit  wavelength-dependent  extinction 
characteristics.  Furthermore,  in  some  applications  the  polarizers  are 
positioned  by  the  attitude  of  the  viewer's  head  and  tilting  the  head  reduces 
the  maximum  possible  extinction.  (Extinction  varies  as  the  square  of  the 
cosine  of  the  angle  between  the  polarization  axes  of  the  display  and 
spectacle  polarizers.)  Thus,  if  a  viewer  tilts  his  head,  some  of  the  right- 
eye  signal  will  be  seen  by  the  left  eye  and  vice  versa,  thereby  producing 
annoying  "ghost  images."  In  systems  utilizing  circularly  polarized  filters, 
this  problem  is  eliminated. 

Both  projection  and  direct-viewing  applications  have  been  developed,  the 
most  common  projection  application  being  3-D  motion  pictures.  Projection 
systems  must  be  based  upon  reflecting  surfaces  that  do  not  significantly 
diminish  the  polarization  of  the  left-  and  right-eye  images.  In  one  direct- 
viewing  display  system,  a  sector  disk  composed  of  alternating  segments  of 
orthogonal  polarizers  is  positioned  between  the  display  screen  and  the 
viewer.  This  sector  disk  is  rotated  synchronously  with  the  field  rate  of  the 
display  so  that  alternate  fields  are  viewed  by  the  two  eyes.  This 
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application  has  the  disadvantage  of  producing  a  variable  extinction  ratio  of 
the  binocular  images  by  requiring  that  the  polarizers  rotate  into  position  in 
front  of  the  display  screen.  Cross-talk  is  relatively  high  in  this  type  of 
display. 

The  polarization  stereogram  technique  of  image  separation  imposes  no 
constraints  on  the  resolution,  color,  speed,  or  number  of  viewers,  though 
brightness  of  the  displayed  image  is  reduced  significantly  by  the  addition  of 
the  filter.  As  in  other  stereo  systems,  full  parallax  is  not  maintained,  and 
the  system  requires  the  user  to  wear  spectacles.  Because  of  its  relative 
independence  from  the  display  medium,  the  technique  can  provide  added 
capability  to  existing  systems.  For  computer  displays,  the  burden  levied  on 
the  computer  system  is  essentially  that  of  doubling  the  number  of  displayed 
images  and  the  computations  to  create  them.  Considerations  for  applications 
involving  real-time  displays  are  addressed  in  the  section  on  Field  Sequential 
techniques. 

4. 1.1.5  Field  Sequential 

Stereo  displays  based  on  field  sequential  television  technology  have 
been  developing  rapidly  during  the  last  several  years.  This  technology 
presents  left-eye  and  right-eye  views  alternately  on  the  TV  screen  and 
permits  each  eye  to  see  its  own  view.  Broadcast  TV  lends  itself  readily  to 
this  type  of  display  because  of  the  way  in  which  the  image  is  written  to  the 
screen.  Broadcast  TV  presents  30  frames  per  second,  with  each  frame 
consisting  of  two  interlaced  fields  of  scan  lines.  The  first  field  consists 
of  the  odd-numbered  scan  lines  and  the  second  field  consists  of  the  even- 
numbered.  Thus,  it  is  possible  to  present  both  eye  views  in  each  frame  by 
making  the  first  field  the  left  eye  view  and  the  second  field  the  right  eye 
view. 


The  problem  then  becomes  one  of  image  separation,  which  resolves  itself 
to  one  of  synchronizing  left-  and  right-eye  masks  to  the  field  rate  of  the 
TV.  Numerous  methods  have  been  developed  for  accomplishing  this  image 
separation,  but  they  all  suffer  from  several  failings.  Chief  among  these  are 
flicker  and  cross-talk.  Although  the  standard  TV  frame  rate  is  30  Hz,  the 
interlaced  field  rate  is  60  Hz.  In  field  sequential  stereoscopic  TV,  each 
eye  sees  only  one  field,  so  its  effective  rate  is  only  half  that  of  the 
normal  TV  field  rate,  i.e.  30  Hz.  Alternation  at  30  Hz  can  produce  annoying 
flicker.  One  solution  is  to  double  the  usual  frame  rate  to  60  Hz,  that  is, 
to  produce  a  60  Hz  noninterlaced  system  or  by  developing  a  120  Hz  field-rate 
monitor.  The  monocular  frame  rate  will  then  be  60  Hz,  but  the  resultant 
technology  does  not  comply  with  NTSC  TV  standards. 

The  other  major  problem  is  cross-talk  between  the  left-eye  and  right-eye 
images.  One  source  of  cross-talk  is  the  persistence  of  the  phosphors  that 
form  the  image.  If  the  phosphors  cannot  be  switched  off  quickly  enough,  some 
of  the  left-eye  image  may  still  be  visible  on  the  screen  when  the  right-eye 
shutter  is  opened.  This  is  not  as  pronounced  a  problem  with  60  Hz 
television,  but  it  is  a  consideration  in  120  Hz  monitors. 
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Field  sequential  TV  displays  have  another  characteristic  that  detracts 
from  the  veracity  of  their  stereoscopic  image  quality,  one  is  that  is  shared 
by  other  technologies  that  separate  monocular  images  artificially.  When  the 
viewer  moves  laterally  with  respect  to  the  display  screen,  the  motion 
parallax  induced  by  this  movement  is  opposite  that  which  would  occur  if  the 
viewer  moved  while  examining  a  real  three-dimensional  object.  When  examining 
a  real  3-D  object,  a  rightward  movement  would  produce  an  apparent  leftward 
movement  of  the  object,  i.e.  more  of  the  right  side  of  the  object  would  be 
exposed.  However,  rightward  movement  of  the  viewer  does  not  expose  more  of 
the  right  side  of  the  synthesized  stereoscopic  object,  and  the  visual  system 
interprets  this  as  corresponding  rightward  movement  of  the  displayed  object. 
Thus,  motion  parallax  produced  by  viewer  movement  is  of  the  wrong  polarity, 
which  can  be  a  source  of  annoyance. 

Two  major  types  of  field  sequential  TV  are  recognized:  those  that  have 
the  active  image-separation  element  at  the  image  source,  and  those  in  which 
active  separation  is  accomplished  at  the  viewer.  Both  types  utilize  the 
polarization  technique  described  above.  Both  require  that  the  viewer  wear 
spectacles,  but  the  latter  has  the  additional  requirement  that  the  viewer's 
spectacles  be  linked  to  the  display,  usually  by  wires. 

When  the  monocular  images  are  separated  at  the  source,  i.e.  at  the 
display,  the  viewer  need  only  wear  passive  polarizer  spectacles.  The  TV 
screen  is  viewed  through  an  electro-optic  system,  such  as  a  Lead- Lanthanum- 
Zirconium-Titanate  (PLZT )  shutter.  Part  of  the  PLZT  shutter,  which 
incorporates  a  plane  polarizer  and  the  PLZT  crystal,  is  positioned  just  in 
front  of  the  TV  screen.  When  an  electrical  potential  is  impressed  across  the 
PLZT  crystal,  it  rotates  the  plane  of  polarization  of  an  incident  light  very 
rapidly  through  90  degrees.  If  a  polarizer  of  the  same  orientation  as  that 
near  the  TV  screen  is  worn  over  the  right  eye,  then  the  right  eye  will 
receive  light  from  the  screen  when  the  PLZT  crystal  is  in  the  ground  state, 
but  it  will  not  receive  light  from  the  screen  when  the  PLZT  crystal  is 
excited. 

Conversely,  if  a  polarizer  orthogonal  to  that  near  the  TV  screen  is  worn 
over  the  left  eye,  then  the  left  eye  will  receive  light  from  the  screen  only 
when  the  PLZT  crystal  is  in  the  excited  state.  By  synchronizing  the 
excitation  of  the  PLZT  crystal  to  the  two  fields  of  each  TV  frame,  it  is 
possible  to  present  one  field  of  scan  lines  to  the  left  eye  and  the  other 
field  to  the  right  eye.  If  the  images  produced  by  the  scan  lines  are  two 
perspective  images  of  the  same  object,  it  is  possible  to  produce  a 
stereoscopic  image  of  that  object. 

This  type  of  field-separation  system  has  the  advantage  of  requiring  that 
the  viewer  wear  only  passive  elements  of  the  PLZT  shutter  system,  i.e. 
orthogonal  plane  polarizing  spectacles.  The  currently  available  Tektronix 
system  adds  about  $12,000  to  the  cost  of  a  graphics  workstation. 

The  second  type  of  field-separation  system  mounts  the  entire  PLZT 
shutter  system  on  the  viewer.  Each  spectacle  consists  of  a  sandwich  of 
crossed  polarizers  with  PLZT  crystal  filling.  The  principle  of  operation  is 
the  same  as  that  described  above,  but  this  embodiment  of  field  sequential  TV 
suffers  the  disadvantages  of  requiring  a  wire  link  to  the  viewer  and  a 
separate  PLZT  shutter  system  for  each  viewer.  Several  Japanese  companies 
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sell  inexpensive  versions  of  this  system,  and  one  offers  a  3-D  camcorder  for 
about  $2000. 

As  discussed  above,  the  polarization  stereogram  method  adds  a  stereo 
capability  to  an  existing  display  system.  Equipment  is  commercially 
available  that  can  adapt  high  performance  computer  graphics  displays  to 
stereo  field  sequential  devices.  Given  sufficient  computational 
capabilities,  any  battle  management  application  suitable  for  computer  display 
can  enjoy  the  added  benefit  of  stereo  at  the  cost  of  some  image  brightness, 
possible  bleed  through  of  images,  and  lack  of  full  motion  parallax.  For 
situation  assessment  activities,  these  drawbacks  are  not  a  major  problem. 
However,  the  potential  for  visual  fatigue  and  the  need  to  wear  passive 
glasses  for  stereo  viewing  would  be  a  persistent  cause  of  user  annoyance.  In 
mission  planning  applications,  the  capabilities  provided  by  3D  could  be 
particularly  impressive.  The  most  serious  problem,  however,  might  prove  to 
be  the  polarity  of  motion  parallax.  While  systems  presenting  the  planner's 
view  of  the  battlefield  might  not  be  significantly  affected,  the  utility  of 
those  portraying  the  pilot's  view  would  suffer  markedly.  However,  until 
further  human  factors  research  is  performed  on  stereo  systems,  the  actual 
extent  of  their  utility  versus  difficulty  of  use  cannot  be  accurately  gauged. 

4.1.2  Source  Separation  of  Images 

4. 1.2.1  Parallax  Panoramagram 

In  this  technology,  several  images  of  a  scene,  each  taken  from  a 
slightly  different  perspective,  are  presented  on  the  same  display  surface. 
Each  eye  is  permitted  to  see  only  one  image  of  the  scene,  that  which  is  from 
its  own  perspective,  and  from  these  two  disparate  images  the  visual  system 
synthesizes  a  perception  of  depth.  Early  examples  of  parallax  panoramagrams 
used  slit  plates  between  the  display  plane  and  the  viewer.  The  slits  were 
aligned  vertically  and  served  to  expose  single  images  to  each  eye.  The  most 
common  examples  of  static  parallax  panoramagrams  are  three-dimensional 
postcards.  Several  attempts  at  developing  3-D  television  used  this 
technology.  For  C3I  applications,  this  limits  the  number  of  viewers  to  one, 
and  the  position  of  the  viewer's  head  is  very  important. 

One  advantage  of  parallax  panoramagrams  is  that  they  are 
autostereoscopic,  requiring  no  user-worn  aids  to  impart  a  sense  of  depth. 
However,  like  many  autostereoscopic  displays,  this  technique  can  produce 
zones  of  stereoimagery  separated  by  zones  of  pseudoscopic  imagery. 
Pseudoscopic  imagery  occurs  whenever  the  relationship  of  viewer,  slit  plate, 
and  display  screen  causes  the  left  eye  to  see  the  right-eye  image  and  vice 
versa.  This  problem  can  be  corrected  by  the  use  of  "dead"  or  black  zones 
between  the  left  and  right  eye  images.  Another  limitation  of  this  technology 
is  that  the  number  of  stereoscopic  depth  planes  that  can  be  produced  depends 
on  the  number  of  images  that  can  be  displayed  across  the  screen  and 
successfully  hidden  by  the  slit  plate.  In  the  simplest  examples,  only  three 
or  four  depth  planes  are  produced.  For  still  images  captured 
photographically,  this  is  not  a  problem.  For  dynamic  computer  images, 
however,  this  causes  an  apparent  flattening  of  objects  in  the  field  of  view, 
such  that  objects  look  like  cardboard  cutouts.  Although  resolution  and  color 
are  not  affected  by  the  technique,  the  limitations  on  the  depth  of  objects 
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and  limitations  on  viewing  zones  restrict  its  use  in  mission  planning 
operations. 

Finally,  most  parallax  panoramagrams  use  one-dimensional  image- 
separation  elements  and,  hence,  lack  full  motion  parallax.  As  a  viewer  moves 
his  head  with  respect  to  the  display  surface,  the  stereoscopic  Images  do  not 
occlude  (reveal)  previously  visible  (hidden)  surfaces  as  occurs  in  true 
three-space.  This  is  frequently  interpreted  by  the  visual  system  as  in-phase 
movement  of  objects  in  the  scene,  so  these  objects  appear  to  move  in  the  same 
direction  as  the  head  movements.  This  can  be  an  annoying  artifact  of 
parallax  panoramagrams,  as  with  the  other  systems  already  described. 

4. 1.2.2  Parallactiscope 

The  parallactiscope  is  essentially  a  temporal  model  of  the  parallax 
panoramagram,  which  uses  a  series  of  slits  to  ensure  that  each  eye  sees  only 
its  perspective  view  of  a  three-dimensional  image.  The  system  consists  of  a 
CRT  on  which  are  presented  several  perspective  views  of  the  same  object  and  a 
moving  slit  that  exposes  sequential  perspective  views  of  that  object.  The 
CRT  is  synchronized  with  the  moving  slit  so  that  the  left  eye  only  sees  the 
object  from  its  perspective,  and  likewise  for  the  right  eye.  In  one  version, 
built  by  Homer  Tilten  of  Arizona,  the  "slit"  is  a  thin  vertical  strip  of  1/4- 
wave  plate  that  moves  across  the  face  of  the  CRT  in  front  of  a  sheet  of 
polarizing  material.  The  viewer  wears  spectacles  containing  polarizers  that 
are  orthogonal  to  the  polarizer  in  front  of  the  screen.  Thus,  images  on  the 
face  of  the  CRT  are  not  perceived  by  the  viewer  except  where  the  1/4-wave 
plate  renders  them  visible. 

Due  at  least  in  part  to  the  fact  that  the  moving  slit  technology  of  the 
parallactiscope  is  mechanical,  the  effective  frame  rate  of  the  device  is  slow 
enough  to  produce  flicker.  This  could  be  eliminated  by  using  electro-optical 
slits,  such  as  liquid  crystal  or  PLZT,  but  a  more  difficult  problem  to  solve 
is  the  limitation  imposed  on  the  complexity  of  the  displayed  image  by  the 
requirement  to  produce  sequential  perspective  images  for  both  eyes.  The 
greater  the  number  of  views,  the  better  the  perception  of  depth.  However,  as 
the  effective  frame  rate  increases,  the  time  during  which  the  perspective 
image  can  be  generated  decreases.  At  the  moment,  the  system  is  limited  to 
presenting  relatively  simple  images  such  as  3-D  waveforms  and  helical  coils 
to  a  single  viewer  and  does  not  provide  full  motion  parallax.  Even  with 
technological  advances  in  available  bandwidth,  the  restrictive  viewing  zone 
severely  restricts  the  application  to  battle  management  problems. 

4. 1.2. 3  Lenticular  Screen 

To  circumvent  the  technical  problems  involved  in  the  development  of  ever 
finer  slit  plates,  the  lenticular  screen  was  developed.  In  this  technology, 
each  eye  is  prohibited  from  viewing  its  partner's  image  by  optical  means 
rather  than  by  occlusion.  There  are  many  different  arrangements  of 
lenticular  screen  displays  but,  in  principle,  they  all  operate  in  the  same 
fashion.  Like  the  slit  plate,  the  lenticular  screen  must  be  positioned  very 
precisely  with  respect  to  the  multiple  images  on  the  display  surface.  When 
properly  registered,  the  lenticular  screen  delimits  the  area  of  the  display 
surface  visible  to  each  eye  by  refraction. 
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Because  of  the  interocular  distance,  the  line  of  sight  from  each  eye 
intersects  the  same  lens  surface  at  a  slightly  different  angle.  This  angular 
difference  causes  different  parts  of  the  display  surface  to  be  imaged  in  the 
two  eyes,  effecting  binocular  image  separation.  Like  the  slit  plate 
technology,  the  lenticular  screen  type  of  display  is  limited  by  the  number  of 
images  that  can  be  successfully  masked.  Recent  developments  have  permitted  a 
much  greater  number  of  multiple  representations  to  be  presented,  but  the 
technology  still  suffers  from  the  problems  of  viewer  position  and  lack  of 
motion  parallax.  At  this  point  in  their  development,  neither  lenticular 
screen  nor  parallacti scope  technology  is  robust  enough  to  be  fielded.  Work 
is,  however,  continuing  in  both  areas  and  the  capabilities  continue  to 
expand. 


4. 1.2.4  Vertical  Parallax 

Vertical  parallax  can  provide  a  compelling  impression  of  depth,  but  by 
itself  does  not  support  stereopsis.  It  is  mentioned  here  because  of  the 
recent  development  of  technologies  based  on  vertical  parallax  that  purport  to 
convey  depth  information.  In  the  vertical  parallax  display  system,  two 
images  of  the  same  three-dimensional  scene,  each  obtained  from  a  slightly 
different  vertical  position,  are  presented  sequentially.  Like  horizontal 
parallax,  vertical  parallax  can  selectively  occlude  or  reveal  features  of  a 
three-dimensional  scene.  When  two  images  obtained  from  different  elevations 
are  presented  alternately,  features  that  are  apparent  in  the  image  obtained 
from  the  higher  elevation  seem  to  disappear  when  the  image  obtained  from  the 
lower  elevation  is  presented. 

A  3-D  TV  system  based  upon  vertical  parallax,  called  Visidep,  has  been 
developed.  It  uses  a  pair  of  TV  cameras  mounted  one  above  the  other  to 
produce  images  with  vertical  parallax.  Initially,  this  system  was  designed 
to  present  the  two  images  in  alternation,  one  frame  at  a  time,  but  that  was 
found  to  produce  too  much  flicker.  Instead,  a  limited  number  of  frames 
obtained  from  one  camera  are  interlaced  with  those  from  the  other  camera. 
This  reduces  flicker,  but  it  produces  apparent  vertical  jumps  of  objects  in 
the  scene.  The  Visidep  system  is  not  a  true  stereoscopic  display  system,  and 
its  inherent  image  motion  will  probably  constrain  its  use  as  a  3-D  display 
system. 


4.2  Volumetric  Techniques 

In  contrast  to  stereoscopic  systems  which  present  two  perspective  views 
of  a  scene  in  a  reconstruction  of  binocular  vision,  volumetric  techniques 
attempt  to  create  a  three-dimensional  image.  Methods  for  doing  this  vary 
widely  and  have  achieved  some  commercial  success.  Because  of  the  spatial 
nature  of  their  images,  volumetric  systems  in  many  cases  have  the  advantage 
over  most  stereo  systems  of  being  viewable  by  multiple  observers  without  the 
need  for  user-worn  aids.  Section  4.2.1  describes  various  holographic 

techniques.  Section  4.2.2  discusses  varifocal  mirrors,  and  Section  4.2.3 
addresses  several  other  systems. 

4.2.1  Holograms 

Holograms  generally  provide  static  rather  than  dynamic  views  wh’ch  can 
be  viewed  by  more  than  one  observer  and  require  no  observer-worn  image- 
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separation  hardware.  Usually  holographic  images  are  monochromatic  and  rely 
only  on  differences  in  brightness  to  render  images.  Many  holograms  require 
laser  reconstruction  of  photographic  images,  which  renders  them  useless  for 
real-time  display  and  potentially  dangerous,  although  safe  low-wattage  lasers 
will  produce  adequate  holograms.  Visual  fatigue  should  not  be  a  problem,  as 
accommodation  and  convergence  signals  are  not  antagonistic  while  viewing 
holograms,  and  glare  from  competing  ambient  sources  is  not  a  problem. 
Because  of  their  static  nature  and  due  to  the  processing  required  to  create 
holographic  images,  their  use  in  real-time  battle  management  applications  is 
not  expected  in  the  foreseeable  future.  Research  is  continuing,  however,  in 
many  areas  of  holography.  The  brief  descriptions  of  different  holographic 
techniques  provided  below  is  intended  to  acquaint  the  reader  with  the 
diversity  of  methods  being  explored,  describe  some  of  their  limitations  and 
identify  promising  areas  of  interest. 

4.2. 1.1  White-Light  Holograms 

White-light  or  rainbow  holograms,  first  developed  by  Stephen  Benton 
(formerly  of  Polaroid  Corporation  and  now  at  MIT),  are  static  images  that  do 
not  require  laser  light  for  reconstruction.  The  holograms  are  recorded  by 
exposing  only  a  thin  vertical  strip  of  photographic  plate  at  a  time. 
Inexpensive  holograms  can  be  made  by  either  embossing  mylar  with  a  nickel 
holographic  image  or  by  the  dichromatic  method,  in  which  transparent 
holograms  are  made  from  the  original  opaque  image. 

White-light  holograms  provide  genuine  horizontal  parallax,  but  not 
vertical  parallax.  Thus,  vertical  movement  of  the  observer  produces 
perspective  distortion.  Furthermore,  because  these  displays  disperse  white 
light  in  a  manner  similar  to  a  diffraction  grating,  vertical  motion  also 
produces  changes  in  chromaticity  of  the  holographic  image.  Image  resolution 
and  contrast  are  both  typically  rather  low. 

4.2. 1.2  Alcove  Holograms 

Alcove  holograms  are  a  recent  development  of  the  Media  Labs  at  MIT. 
They  comprise  a  concave  180-degree  photographic  film  that  produces  a 
panoramic  holographic  stereogram.  The  viewer  perceives  the  holographic 
object  as  being  imaged  in  space  beyond  the  cylindrical  film  surface.  It  is 
possible  to  design  the  alcove  hologram  so  that  stereoscopic  objects  appear  to 
move  as  the  viewer  scans  the  surface  of  the  film.  The  alcove  hologram  has 
many  of  the  characteristics  of  the  white-light  hologram  and,  thus,  shares 
many  of  its  disadvantages.  This  technology  shows  promise  for  future 
development,  but  for  the  moment,  development  of  this  technology  is  proceeding 
slowly. 


4.2. 1.3  Multiplex  Holograms 

This  is  the  commercial  name  for  the  embodiment  of  a  type  of  hologram 
called  the  integral  or  Cross  hologram  [after  Dr.  Cross  who  was  one  of  the 
first  (with  David  Schmidt)  to  learn  how  to  produce  these  holograms].  The 
Cross  hologram  is  a  cylinder  that  is  illuminated  from  below  with  either  white 
light  or  monochromatic  light  (different  technologies).  Recorded  on  the 
hologram  are  multiple  images  of  an  object  taken  at  different  times.  If  the 
object  moved  during  the  period  when  the  multiple  images  were  being  recorded. 


25 


the  observer  sees  the  object  move  as  the  cylinder  is  rotated.  This 
technology  is  commercially  available  through  a  very  limited  number  of 
suppliers,  and  at  a  cost  considerably  greater  than  white-light  holograms. 

4.2. 1.4  Synthesized  Holograms 

Synthesized  or  computer-generated  holograms  may  be  the  hard-copy  of  the 
future,  but  it  is  unlikely  that  they  will  constitute  a  major  thrust  in  three- 
dimensional  display  technology.  Current  problems,  such  as  the  time  required 
for  synthesis  and  the  resolution  limitations  of  synthesizers,  will  probably 
yield  to  technological  improvements.  However,  the  inherent  limitations  of 
holograms,  e.g.  monochrome,  static  images,  will  continue  to  limit  the 
applicability  of  these  devices. 

4.2. 1.5  Composite  Holograms 

Composite  holograms  are  a  collection  of  many  small  subholograms,  each 
with  a  different  perspective  view  of  the  object.  Because  each  eye  views  a 
different  subhologram,  the  object  is  seen  in  depth  from  a  particular 
viewpoint.  With  subholograms  larger  than  about  two  millimeters,  noticeable 
jumps  in  perspective  occur  during  head  movements.  The  composite  hologram 
usually  lacks  vertical  perspective  and  suffers  the  disadvantages  of  other 
white-light  holograms. 

4.2. 1.6  Combination  Holograms 

Holograms  can  be  made  by  combining  the  synthetic  and  composite  types. 
In  this  technology,  a  computer  is  used  to  generate  a  series  of  perspective 
views  of  an  object  on  the  display.  Each  view  is  recorded  on  a  photographic 
plate,  just  as  it  would  be  in  a  composite  hologram.  The  combination  hologram 
exhibits  the  same  jumps  of  perspective  seen  in  the  composite  hologram  and  has 
the  reduced  resolution  of  the  synthesized  hologram.  One  product  that  could 
be  produced  by  this  combination  of  technologies  is  the  full-color  hologram. 
Currently  in  the  early  stages  of  development,  the  full  color  hologram 
represents  the  nearly  insurmountable  problem  of  registering  three  synthetic 
holograms  on  the  same  holographic  plate  -  one  each  for  the  red,  green,  and 
blue  separations  of  a  color  image. 

4. 2. 1.7  Holographic  Cinema 

This  technology  involves  the  generation  of  projection  holograms  on 
curved,  reflecting  surfaces.  Very  little  is  known  about  this  technology, 
although  there  has  been  interest  in  the  Soviet  Union  for  some  time.  A  major 
drawback  seems  to  be  the  necessity  of  having  the  viewer  centered  with  respect 
to  the  projection  system,  which  means  that  only  one  person  can  perceive  the 
stereo  image  at  a  time. 

4.2. 1.8  Pepper's  Ghost 

This  is  a  commercially  available  embodiment  of  holographic  technology 
that  is  used  at  The  Haunted  Mansion  in  Disneyland.  The  display  consists  of  a 
volume  within  which  three-dimensional  figures  are  seen  to  revolve  about  their 
own  axes  and  to  rotate  through  the  volume.  The  technology  was  developed  by  a 
company  called  White  Light  Works  which  is  involved  in  other  holographic 
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projects.  Although  the  technology  Involved  is  proprietary,  we  can  guess  that 
each  of  the  revolving  figures  may  be  produced  by  overhead  or  Internal 
illumination  of  multiplex  holograms. 

4.2.2  Varifocal  Mirror 

The  varifocal  mirror  concept  uses  the  distortions  produced  by  image 
reflection  to  synthesize  a  volumetric  view  of  features  displayed  on  a  two- 
dimensional  screen.  Several  different  types  have  been  developed,  each  of 
which  has  advantages,  but  only  one  type  has  been  brought  to  the  stage  of 
commercialization.  Displays  based  on  varifocal  mirrors  require  low  ambient 
illumination  for  best  performance. 

4.2.2. 1  Vibrating  Mirror 

SpaceGraph  is  the  commercial  name  of  a  stereodisplay  system  based  on  the 
vibrating-mirror  embodiment  of  the  varifocal-mirror  concept.  It  uses  a 
silvered  membrane  that  is  oscillated  sinusoidally,  much  like  the  cone  of  a 
speaker  driven  by  a  pure  tone.  The  membrane  oscillates,  alternating  between 
a  concave  mirror  and  a  convex  mirror  thirty  times  per  second.  Concave 
mirrors  make  objects  appear  closer  than  they  are  and  convex  mirrors  make  them 
appear  to  be  further  away.  Thus,  a  two  dimensional  image  reflected  from  the 
surface  of  the  oscillating  silvered  membrane  would  appear  to  sweep  out  a  path 
in  space,  producing  a  virtual  three  dimensional  image. 

The  SpaceGraph  system  uses  this  principle  to  generate  depth.  The  user 
views  the  face  of  a  CRT  reflected  in  the  oscillating  mirror.  Points  of  light 
are  displayed  on  the  face  of  the  CRT  synchronously  with  the  vibrations  of  the 
mirror.  Points  that  are  displayed  when  the  mirror  is  convex  will  appear  to 
be  further  away  from  the  viewer  than  points  that  are  displayed  when  the 
mirror  is  concave.  It  is  possible  to  construct  a  representation  of  a  three- 
dimensional  object  by  displaying  specific  pixels  when  the  mirror  is  in  the 
appropriate  phase  of  its  vibration. 

This  technology  is  ideal  for  presenting  monochrome  images  of  partially 
transparent  objects,  such  as  models  of  molecules  or  a  volume  of  airspace,  but 
it  cannot  in  its  current  form  be  used  to  render  solid  surfaces.  Only  a 
limited  number  of  pixels  can  be  addressed  when  the  mirror  is  in  a  particular 
position  which  limits  the  total  number  of  points  that  can  be  used  to 
represent  an  object.  Also,  the  center  of  the  mirror  is  constrained  to  move 
through  a  relatively  short  vibratory  path  because  of  acoustic  considerations, 
which  limits  the  range  of  depths  that  can  be  rendered.  The  acoustic 
limitations  of  the  system  also  constrain  the  refresh  rate  for  each  pixel, 
resulting  in  noticeable  flicker. 

The  3D  image  presented  by  SpaceGraph  is  a  very  compelling  one.  No  aids 
are  required  to  be  worn  by  the  viewer,  and  the  number  of  viewers  is  limited 
primarily  by  the  relatively  small  display  space.  Unlike  other  systems  in  or 
near  commercial ization,  SpaceGraph  preserves  full  motion  parallax  by  actually 
displaying  the  image  in  space  rather  than  simulating  spatial  perception. 
Another  consequence  of  the  technique  is  robustness:  differences  in  viewing 
capabilities  of  individuals  are  not  aggravated  by  artificially  imposed  values 
such  as  interocular  distance.  Issues  of  resolution  and  real-time  processing 
speeds  can  be  addressed  more  by  the  computational  power  of  the  host  machine 
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than  by  the  display  technique  itself.  Problems  relating  to  phospher  speeds, 
however,  need  to  be  resolved  before  color  can  be  supported. 

In  summary,  the  use  of  the  vibrating  mirror  technology  in  C3I 
applications  is  promising.  In  the  short  term,  situation  assessment  and  data 
visualization  tasks  appear  to  be  particularly  appropriate.  Eventually,  its 
use  in  aspects  of  operational  mission  planning,  especially  for  support 
functions  to  the  mission  planner,  can  be  expected.  However,  SpaceGraph  has 
been  under  development  for  about  a  decade,  with  limited  recent  progress. 
Until  the  inherent  limitations  of  this  approach  are  seriously  addressed 
(acoustic  limitations  of  the  mirror,  refresh  rate,  color,  computational  power 
needed  to  handle  increased  speeds,  etc.),  the  application  of  the  technology 
will  be  severely  limited. 

4. 2. 2. 2  Oscillating  Mirror 

Displays  have  been  developed  that  rely  on  a  pair  of  oscillating  mirrors 
(instead  of  a  single  mirror  like  SpaceGraph)  to  generate  the  third  visual 
dimension.  Each  requires  the  user  to  view  an  image  of  a  display  screen  that 
has  been  reflected  from  a  pair  of  mirrors.  The  mirrors  oscillate  with 
respect  to  each  other,  so  the  optical  path  length  from  the  display  screen  to 
the  viewer  varies.  Like  the  SpaceGraph,  this  system  lights  pixels 

synchronously  with  the  location  of  the  oscillating  mirrors  to  produce  depth. 
Also  li^e  SpaceGraph,  this  system  is  capable  of  producing  only  partially 
transparent  images,  not  surfaces.  However,  because  the  technique  does  not 
take  advantage  the  way  SpaceGraph  does  of  the  large  change  in  magnification 
that  occurs  with  nonplanar  mirrors,  the  range  of  depths  that  can  be  produced 
by  this  system  is  greatly  reduced.  While  the  use  of  dual  oscillating  mirrors 
eliminates  some  of  the  acoustic  problems  of  the  vibrating  mirror  technique, 
implementation  and  synchronization  of  the  movement  of  the  two  mirrors  pose 
other  electro-mechanical  difficulties.  Performance  is  reduced  by  the  use  of 
planar  mirrors,  and  developmental  work  on  the  method  has  not  been  as 
consistent.  As  a  result,  the  technology  is  not  as  mature  as  vibrating 
mirrors. 

4.2.3  Other  Volumetric  Techniques 

4. 2. 3.1  Dual  Laser  Beam  Fluorescence 

This  is  an  interesting  technology  that  generates  volume-filling 
autostereoscopic  images.  The  display  medium  is  a  transparent,  gas-filled 
vessel  within  which  the  stereo  images  appear.  The  gas  that  fills  the  volume 
has  the  characteristic  that  it  fluoresces  when  excited.  Two  laser  beams  are 
directed  into  the  gas  by  a  two-dimensional  mirror  system  that  can  steer  each 
laser  beam  independently.  The  power  of  each  laser  is  too  low  to  excite  the 
gas  to  fluorescence,  but  the  combined  power  of  both  lasers  is  sufficient  to 
produce  fluorescence.  Thus,  the  gas  will  only  fluoresce  where  the  laser 
beams  intersect.  By  steering  the  mirror  systems  and  by  pulsing  the  lasers, 
it  is  possible  to  energize  loci  within  the  gas  that  describe  an  object.  Wher 
this  is  done,  a  three-dimensional  image  of  that  object  appears.  T'  s 
possible  to  generate  dynamic  as  well  as  static  3-D  objects  with  chis 
technology. 
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Several  Interrelated  problems  hamper  the  continued  development  of  this 
system.  If  the  persistence  time  of  the  gas  Is  reduced  so  that  dynamic 
objects  do  not  produce  smeared  images,  the  refresh  rate  becomes  very  high. 
This  is  limited,  in  part,  by  the  laser-steering  system.  The  excitation 
images  produced  by  this  system  are  monochromatic  and  they  must  be  viewed 
through  the  gaseous  atmosphere  of  the  display  volume.  These  problems  may  be 
alleviated  by  continued  development,  eventually  providing  an  excellent  medium 
for  multi-viewer,  full  parallax  viewing  of  dynamic  3D  images,  but  it  is 
unlikely  that  these  displays  will  become  commonplace  in  the  near  future. 

4. 2. 3. 2  Spinning  Mirror 

The  spinning  mirror  system  is  another  means  of  using  a  mirror  to  sweep  a 
volume  in  visual  space.  This  system,  developed  at  MIT  several  years  ago, 
uses  a  mirror  rotating  about  its  center  on  an  axis  that  is  not  normal  to  the 
reflecting  surface.  Images  reflected  from  the  mirror  describe  an  arc  as  the 
mirror  turns,  so  that  a  two-dimensional  array  of  pixels  viewed  in  reflection 
from  the  mirror  will  describe  a  three-dimensional  space.  By  addressing  the 
pixels  of  a  two-dimensional  display  screen  synchronously  with  the  angle  of 
rotation  of  the  mirror,  it  is  possible  to  generate  volumetric  images  of 
partially  transparent  objects.  Like  displays  based  on  the  varifocal  mirror 
concept,  spinning  mirror  displays  are  subject  to  flicker.  However,  because 
there  is  no  oscillatory  or  vibrational  mode  to  spinning  mirror  displays, 
flicker  can  be  eliminated  by  raising  rotational  speed.  The  limiting  factors, 
as  described  above,  then  become  the  pixel  address  rate  and  phosphor 
persistence.  The  comments  regarding  the  applicability  of  vibrating  mirror 
technology  to  battle  management  tasks  (pending  resolution  of  specific 
technical  issues)  apply  here  as  well. 

4. 2. 3. 3  Spinning  LED 

The  spinning  LED  display  is  similar  in  concept  to  the  spinning  mirror 
display  except  that  in  the  former,  the  display  per  se,  rather  than  an  image 
of  the  display,  rotates  about  its  center.  An  array  of  LEDs  positioned  on  a 
disk  is  rotated  about  an  axis  normal  to  the  disk  and  individual  LEDs  are 
illuminated  to  produce  outlines  of  objects.  At  any  angle  of  view,  except  the 
axis  of  rotation,  the  viewer  sees  the  LEDs  rotating  in  three-space  because  of 
the  different  distances  to  the  near  and  far  edges  of  the  disk.  This  display 
is  not  capable  of  rendering  surfaces  and  is  best  viewed  in  darkness.  Of 
relatively  limited  utility,  this  display  type  has  not  undergone  significant 
recent  development. 

4. 2. 3. 4  Stacked  Array 

Stacked  array  displays  are  composed  of  some  number  of  independent 
display  surfaces,  one  in  front  of  another,  containing  pixels  that  can  be 
switched  rapidly  from  illuminating  to  invisible.  Nearer  pixels  in  an 
invisible  state  allow  the  illuminated  pixels  on  successive  display  surfaces 
to  be  viewed.  Essentially,  the  pixels  of  the  stacked  arrays  comprise  a 
volume  of  pixels  that  can  be  addressed  to  render  an  object  visible  in  three- 
dimensions.  This  type  of  display  suffers  from  poor  depth  resolution  caused 
by  a  limited  number  of  depth  planes  and  from  reduced  image  resolution  caused 
by  scatter  in  the  intervening  array  planes.  In  its  current  form,  it  is  not 
appropriate  for  battle  management  applications. 
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4.3  Continuing  Developments 

4.3.1  Helmet-mounted  Systems 

One  area  of  increased  interest  and  research  activity  is  that  of  displays 
mounted  in  helmets  for  viewing  by  one  person.  An  appropriate  application  of 
this  technology  in  battle  management  would  be  the  simulation  of  the  pilot's 
view  of  a  flight  scenario.  This  technology  is  also  being  pursued  for  use  in 
telerobotic  situations  in  which  the  actions  of  the  viewer  are  transmitted  to 
a  remote  device  which  performs  the  intended  action.  This  suggests  a  use  in 
the  planning  and  control  of  remotely  piloted  vehicles  as  well  as  manned 
aircraft. 

4.3. 1.1  Dual-Scope 

The  dual-scope,  helmet-mounted  display  has  been  under  development  at 
NASA/Ames.  This  system  uses  binocular,  backlit  LCDs  with  4-bit  gray  scales 
to  produce  the  stereo  images.  Each  LCD  presents  a  two-dimensional  view  of 
the  target  environment,  obtained  from  a  particular  viewpoint.  When  the  two 
LCDs  present  corresponding  images  of  the  same  environment  obtained  from 
different  perspectives,  the  resulting  image  disparity  is  used  by  the  visual 
system  to  synthesize  the  perception  of  stereoscopic  depth.  Among  the 
capabilities  of  this  system  is  the  near-real-time  exploration  of  a 
synthesized  three-dimensional  environment.  This  embodiment  requires  the 
addition  of  a  six-axis  head-position  sensor,  significantly  complicating  the 
device.  At  the  moment  the  NASA/Ames  systems  is  limited  by  hardware 
constraints  to  the  presentation  of  line  drawings. 

Other  helmet-mounted,  dual-scope  displays  have  been  based  on  CRT  rather 
than  LCD  technology.  Like  the  NASA/Ames  system,  these  are  direct-view 
systems  in  which  the  display  screens  are  positioned  so  that  they  are  viewed 
stereoscopical ly  by  the  viewer/wearer.  The  CRT-based  display  system  is 
capable  of  presenting  real-time,  full  color  stereo  images  of  the  three- 
dimensional  environment.  They  currently  suffer,  however,  from  the 
disadvantages  of  excessive  heat,  size,  and  mass  of  the  required  equipment. 

4.3. 1.2  Fiber  Optic 

A  second  type  of  heads-up,  helmet-mounted  stereoscopic  display  system  is 
based  on  coherent  fiber  optic  bundles  that  project  the  separate  left-  and 
right-eye  images  onto  a  partially  silvered  visor.  Because  the  two  images 
produce  retinal  disparity,  the  viewer/wearer  perceives  a  stereoscopic  image. 
Usually,  this  stereoscopic  image  is  perceived  as  originating  at  some  distance 
in  external  space,  and  it  is  seen  superimposed  on  the  external  environment 
viewed  through  the  partially  silvered  visor.  The  utility  of  this  system  is 
diminished  by  its  reliance  on  the  visor,  which,  among  other  things,  degrades 
the  images  of  other  objects  in  the  external  environment.  In  contrast  to  the 
dual  scope  systems,  this  method  allows  the  viewer  to  see  both  the  generated 
and  real  environments.  This  can  provide  a  way  for  a  pilot,  for  example,  to 
compare  what  he  should  be  seeing  with  the  actual  view. 
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4.3.2  Collimated  View  Multiplexing 


This  display  system  uses  a  series  of  vertical  baffles,  conceptually  like 
Venetian  blinds,  to  separate  the  left-  and  right-eye  view  of  a  CRT.  The 
baffles  oscillate  synchronously  with  the  presentation  of  images  on  the  face 
of  the  CRT  to  restrict  the  view  of  those  images  to  the  line-of-sight  for  each 
eye.  Along  each  line-of-sight,  the  image  contains  perspective  information 
consistent  with  that  line-of-sight.  Essentially,  the  system  presents 
alternate  left-  and  right-eye  views,  but  uses  the  oscillating  baffles  to 
restrict  each  eye  to  seeing  only  its  own  view. 

The  necessary  switching  rate  of  the  baffles,  which  must  be  near  the 
refresh  rate  of  the  CRT  to  prevent  flicker,  requires  the  baffles  to  be 
oscillated  electro-optical ly  rather  than  mechanically.  Current  development 
efforts  involve  a  three-dimensional  array  of  pixels  comprising  stacked  layers 
of  two-dimensional  sheets  of  pixels  that  can  be  changed  from  transparent  to 
opaque  electronically.  By  addressing  the  appropriate  pixels  in  each  layer, 
it  is  possible  to  generate  transparent  paths  through  the  stacked  layers. 
When  vertical  lines  of  pixels  in  successive  layers  are  addressed  with 
appropriate  spatial  phase  shifts,  transparent  paths  oblique  to  the  plane  of 
the  layers  can  be  produced.  Alternate  left  and  right  oblique  paths  can  be 
made  transparent  by  changing  only  the  spatial  phase  relationships. 

Collimated  view  multiplexing  is  a  very  new  technology  that  promises  to 
develop  into  a  full-color  autostereoscopic  display  in  the  foreseeable  future. 
However,  at  the  moment,  it  is  little  more  than  a  design  concept  awaiting 
future  refinement. 


5.0  SCENARIO  30  DEMONSTRATION  SYSTEM 

In  addition  to  the  preparation  of  this  Final  Report,  this  task  required 
the  development  of  a  demonstration  system  that  would  demonstrate  the  use  of 
three  dimensional  presentation  in  support  of  a  sample  Air  Force  C3I  battle 
management  function.  This  section  describes  the  intent,  implementation  and 
results  of  the  demonstration  system,  named  SCENARIO. 

5.1  Intent  of  the  Demonstration 

Delivery  of  the  demonstration  system  was  included  as  a  task  requirement 
for  several  reasons.  Most  importantly,  it  was  intended  to  provide  an 
opportunity  for  individuals  to  experience  first-hand  some  of  the  key  points 
presented  in  the  study  report.  This  is  useful  both  in  terms  of  supporting 
the  report  to  people  who  have  read  it  and  in  demonstrating  the  points  of  the 
report  to  people  who  have  not.  In  addition,  the  system  provides  the  user 
with  a  working  example  of  3D  program  code  that  can  serve  as  a  point  of 
departure  for  gaining  experience  in  developing  software  for  a  stereoscopic 
display. 


5.2  Implementation  of  SCENARIO 

(he  3D  Applications  Study  Interim  Meeting,  held  in  Rome,  NY  on  June  8  and 
9  1988,  essentially  marked  the  end  of  the  research  phase  of  the  task  and 
launched  the  demonstration  development  phase.  Implementation  issues 
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discussed  among  team  members  and  RADC  personnel  included  the  identification 
of  3D  features  to  highlight,  development  of  an  outline  of  the  demonstration 
application,  and  evaluation  and  selection  of  hardware  and  software  to  support 
the  demonstration. 

In  discussing  the  strengths  and  weaknesses  of  3D  presentations  for  the 
demonstration,  it  was  noted  that  conventional  perspective  graphics  displayed 
on  a  2D  screen  have  become  so  sophisticated  that  they  can  address  many 
applications  without  requiring  3D.  Three  situations  were  identified, 
however,  in  which  it  was  felt  that  a  3D  system  could  demonstrate  superior 
capabilities. 

The  first  situation  was  decluttering,  in  which  elements  of  graphical  and 
textual  information  displayed  close  together  might  be  disambiguated  by 
viewing  them  in  their  respective  depth  planes.  An  application  of  this 
feature  would  be  air  space  management  where  both  graphics  and  text  must  be 
viewed  for  a  number  of  aircraft.  On  a  2D  display,  the  positions  of  the 
aircraft  can  appear  to  overlap.  The  display  of  corresponding  textual 
information  further  clutters  the  screen.  Displaying  the  data  at  different 
depths  was  felt  to  provide  the  user  with  the  ability  to  distinguish  between 
different  sets  of  information. 

The  second  situation  in  which  3D  was  felt  to  demonstrate  unique 
capabilities  was  in  the  display  of  irregularly  shaped  volumes  and  their 
intersections.  The  depiction  of  regular  geometric  volumes  on  a  2D  display 
provides  the  viewer  with  enough  information  to  imagine  the  shape  of  the  part 
of  the  volume  that  is  unseen.  The  shapes  of  irregular  volumes,  however,  such 
as  irregular  terrain  or  electronic  envelopes  associated  with  electronic 
warfare  equipment,  can  be  very  difficult  to  infer  from  a  2D  representation. 
The  difficulties  are  increased  in  applications  where  it  is  important  to 
perceive  the  intersections  of  such  volumes.  The  use  of  a  display  system  in 
which  the  relative  positions  of  volumes  and  their  surfaces  could  be  perceived 
in  all  dimensions  was  felt  to  give  the  viewer  an  important  added  capability. 

The  third  situation  considered  particularly  appropriate  for  a  3D  display 
was  the  depiction  of  3D  data  over  time.  The  ability  to  view  the  movement  of 
objects  in  a  spatial  scene  was  felt  to  provide  an  entirely  new  dimension  to 
the  user's  perception  of  the  relative  locations  of  objects  and  the  projected 
time  at  which  objects  would  be  at  a  certain  point  in  space. 

An  outline  was  developed  for  the  demonstration  which  was  designed  to 
exhibit  the  capabilities  of  a  3D  display  system  in  the  context  of  an 
operational  Air  Force  task  agreed  upon  by  RADC.  In  order  to  accentuate  the 
user's  involvement  in  the  spatial  environment,  an  interactive  demonstration 
focusing  on  a  tactical  air  support  situation  was  recommended.  To  ensure  that 
the  attention  of  the  viewer  was  focused  on  the  airspace,  it  was  decided  that 
activity  on  the  ground  should  be  kept  to  a  relative  minimum.  The  ability  to 
switch  quickly  between  a  2D  image  and  the  same  image  in  3D  was  considered 
important  for  comparison  purposes.  Eventually,  a  rough  scenario  was 
developed  in  which  the  user's  task  was  to  select  a  set  of  waypoints  from  a 
friendly  airfield  to  an  enemy  target  with  a  minimum  exposure  to  threats,  such 
as  enemy  radars  and  contaminated  areas. 
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Five  hardware  options  for  supporting  the  demonstration  were  then 
considered:  a  large  screen  liquid  crystal  display  offered  by  Greyhawk 
Systems,  the  SpaceGraph  vibrating  mirror  display  offered  by  Bolt,  Beranek  and 
Newman  (BBN),  a  Tektronix  terminal  or  workstation  equipped  with  their  stereo 
shutter,  a  Silicon  Graphics  (SG)  terminal  or  workstation  equipped  with  a 
stereo  shutter  from  StereoGraphics  Corporation,  and  th®~  RADC  Intelligent 
Workstation  or  IWS  (an  Eaton  2000)  equipped  with  the  StereoGraphics  shutter. 
The  Greyhawk  system  was  eliminated  because  it  was  only  a  prototype,  required 
three  minutes  to  refresh  the  screen,  displayed  stereo  images  in  monochrome 
only,  and  had  no  interface  software.  Adapting  the  existing  IWS  w 

considered  because  it  was  the  only  option  that  did  not  require  a  change  in 
the  funding  profile  of  the  task.  In  spite  of  the  availability  of  the 
equipment,  it  was  rejected  as  an  option  due  to  the  degraded  speed  and 
graphics  performance  expected,  the  extended  learning  curve  for  the  software 
development  team  and  the  need  to  either  make  systems  available  to  team 

members  or  port  code  developed  on  other  systems  to  the  IWS.  The  Spacegraph 

by  BBN  had  the  advantages  of  requiring  no  user-worn  aids  and  running  on 
widely  available  PCs.  After  much  discussion,  however,  the  team  concluded 

that  the  limited  size  and  capabilities  of  the  display  (monochrome  wire 
frame),  the  lack  of  proven  interface  software  and  the  fact  that  the  system 
was  still  considered  developmental  by  BBN  outweighed  the  advantages  for  this 
application. 

This  left  a  choice  between  two  stereoscopic  systems  by  Tektronix  and 
Silicon  Graphics.  The  selection  of  Silicon  Graphics  was  the  result  of 
considering  several  factors.  Honeywell  and  Merit,  who  would  be  the  primary 
developers  of  the  software,  both  used  SG  equipment  regularly  and  were  very 
pleased  with  its  performance.  Delivery  of  the  demo  on  a  stereo  Tektronix 

system  would  require  porting  software  developed  without  stereoscopic 
capabilities  from  the  team's  SG  systems.  This  placed  an  unnecessary 

additional  burden  on  the  development  effort.  It  was  also  not  clear  that  the 
stereo  shuttering  technique  adopted  by  Tektronix  performed  as  well  as  the 
StereoGraphics  shutter.  A  final  point  in  favor  of  choosing  Silicon  Graphics 
was  the  availability  of  systems  used  for  demonstration  at  a  reduced  cost.  In 
addition,  the  'C'  programming  language  which  was  determined  to  be  the 
implementation  language  of  choice  due  to  its  power,  portability  and  the 
team's  familiarity  with  it  was  included  on  the  Silicon  Graphics  system  at  no 
additional  charge. 

Once  the  Silicon  Graphics  equipment  was  received,  including  the  monitor 
modified  by  StereoGraphics,  the  monitor  and  supporting  stereo  gear  was  sent 
to  Honeywell  for  development  of  the  interface  software.  Honeywell  then  sent 
the  system  to  Merit  for  integration  with  their  software  and  review  by  IITRI 
and  SRI.  It  was  then  returned  to  IITRI  for  final  modifications  and 
preliminary  review  by  RADC.  Problems  with  SCENARIO  were  cited  and  discussed 
in  detail  at  the  final  technical  meeting.  As  a  result,  some  changes  were 
made  and  additional  features  added  to  the  demonstration  software.  An 
overview  of  the  system  as  delivered  is  provided  below.  More  detail  can  be 
obtained  from  the  SCENARIO  User's/Operator's  Manual. 

As  implemented,  SCENARIO  depicts  the  flight  of  an  aircraft  from  a 
friendly  airfield  to  an  enemy  target.  The  starting  and  ending  points  of  the 
route  are  pre-defined,  and  the  resulting  default  flight  path  is  a  straight 
line  from  airbase  to  target.  Also  defined  in  advance  are  friendly  and  enemy 
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sensor  sites,  often  associated  with  weaponry  positions,  and  clouds  of 
radioactive  contamination  that  move  across  the  scene  as  they  disperse.  The 
task  of  the  user  is  to  modify  the  flight  path  of  the  aircraft  so  as  to 
minimize  exposure  to  both  contamination  and  the  threat  domes  which  indicate 
the  electronic  coverage  of  enemy  sensors.  To  support  this  task,  the  user  is 
able  to  interactively  define  the  route  and  preview  the  aircraft  flying  this 
route  in  near  real-time.  At  any  time,  the  user  can  replan  the  mission  from 
any  waypoint. 

The  demonstration  can  be  run  using  either  of  two  terrain  databases. 
Original  plans  called  for  the  use  of  terrain  data  of  a  50  km  by  70  km  region 
centered  on  the  East/West  German  border,  the  Fulda  Gap.  During  development, 
it  was  determined  that  an  available  terrain  database  of  the  San  Luis  Obispo 
region  of  southern  California  provided  more  detail  of  a  larger  area  (77  km  by 
110  km)  as  well  as  a  more  diverse  range  of  altitudes.  Both  terrain  databases 
were  delivered  with  the  demonstration.  In  both  cases,  the  terrain  is 
displayed  as  an  elevation  shaded  map  in  three  dimensions. 

In  running  the  demonstration,  the  user  has  the  ability  to  select  between 
a  2D  plan  view  of  the  scene,  a  2-1/2  D  perspective  view,  and  a  3D 
stereoscopic  view.  Since  the  demonstration  is  viewed  through  an  active 
stereo  filter,  2-1/2  D  is  achieved  by  setting  the  stereo  disparity  to  0.  The 
2D  plan  view  (or  god's  eye  view)  is  a  special  case  in  which  the  viewpoint  is 
directly  overhead.  The  user  also  has  the  ability  to  move  the  viewpoint  of 
the  observer  to  any  location  relative  to  the  terrain,  including  the  ability 
to  zoom  in  and  out  of  any  scene. 

Because  corresponding  databases  of  actual  cultural  features  were  not 
available  to  the  demonstration  implementation  team,  it  was  necessary  to 
create  entities  that  appear  on  the  landscape.  As  originally  implemented,  the 
only  items  created  were  flat  icons  of  tanks,  cities  and  waypoints,  all  of 
which  floated  above  the  surface  of  the  terrain  and  were  identifiable  only 
from  above.  Upon  review  by  the  team  and  RADC,  it  was  decided  that  3D  icons 
were  necessary  whichfocould  be  scaled  according  to  distance  from  the  viewer, 
identified  from  any  angle  and  that  could  not  be  mistaken  for  actual  scale 
renditions  of  the  items  they  represented.  3D  icons  were  subsequently 
developed  for  missile  launchers,  cities,  waypoints,  airfields  and  the 
aircraft.  It  also  became  apparent  that  the  lack  of  surface  features,  coupled 
with  smooth  shading  of  the  terrain,  made  it  very  difficult  to  discern  the 
actual  surface  of  the  landscape  and  to  judge  the  relative  positions  of 
landscape  features.  Two  efforts  resulted  which  attempted  to  provide 
additional  monocular  depth  cues.  First,  the  algorithm  for  generating  terrain 
color  was  modified  to  allow  for  some  variability  of  color  at  a  given 
altitude.  This  resulted  In  providing  a  sense  of  texture  to  the  terrain  which 
added  some  perspective  cues.  Second,  the  number  of  city  Icons  was  increased 
to  provide  additional  cues  for  perceiving  distance. 

In  its  final  form,  SCENARIO  allows  the  user  to  Interactively 
enable/disable  the  following  classes  of  overlays  at  any  time  during  the 
operation  of  the  demonstration:  cities,  airbases,  command  centers,  targets, 
friendly  forces,  enemy  forces,  contamination  particle  clouds,  threat  domes, 
text,  the  flight  path  of  the  aircraft,  and  a  superimposed  grid  that  indicates 
surface  distance. 
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An  additional  feature,  called  'dynamic  slides'  was  developed  to  allow  the 
user  to  store  multiple  versions  of  display  settings  (such  as  overlay 
selections,  viewer  location,  the  position  of  the  aircraft,  etc.)  to  which  the 
viewer  can  return  with  a  single  keystroke.  These  captured  images,  from  which 
the  user  can  then  resume  execution  of  the  particular  scenario,  were 
originally  intended  for  use  in  demonstrating  the  SCENARIO  system  by  creating 
and  saving  particularly  compelling  views  in  advance.  In  routine  use, 
however,  the  image-saving  capability  has  proven  to  be  an  excellent  tool  for 
storing  points  In  the  scenario  to  which  the  user  may  want  to  return  for 
purposes  of  further  development  or  comparison. 

5.3  Review  of  the  Demonstration 

The  process  of  developing  and  reviewing  the  SCENARIO  demonstration  system 
provided  insights  into  the  use  of  3D  stereoscopic  displays  for  operational 
tasks  that  had  not  been  considered.  Much  of  this  experience  was  directly 
related  to  the  stereoscopic  technology  with  which  the  team  was  working.  In 
particular,  issues  raised  in  reviewing  the  initial  demonstration  related  to 
the  need  of  the  developer  to  understand  the  many  aspects  of  human  viewing 
that  contribute  to  (or  detract  from)  the  perception  of  depth. 

One  example  of  this  was  the  original  decision  to  minimize  the  presence 
and  activity  of  entities  on  the  ground  in  order  to  focus  the  viewer's 
attention  on  the  three  dimensional  airspace.  Upon  reviewing  the 
demonstration,  it  became  apparent  that  the  lack  of  surface  features  resulted 
in  a  poorly  defined  terrain  surface.  This  distracted  the  viewer  and 
detracted  from  the  sense  of  relative  depth  by  eliminating  important  monocular 
depth  cues.  An  effort  was  made  to  reduce  the  effects  of  this  problem  by 
further  populating  the  surface  terrain  with  scalable  (i.e.,  size 
automatically  adjusted  by  the  system  depending  on  distance  from  viewer)  3D 
icons  of  cities  and  targets. 

Additionally,  the  lack  of  surface  features  was  attributed  to  the  use  of  a 
terrain  shading  algorithm  based  on  altitude  rather  than  on  lighting  models. 
The  shading  of  the  terrain  for  SCENARIO  is  accomplished  by  assigning  a  user- 
specified  range  of  240  shades  of  color  to  240  altitude  segments  comprising 
the  range  of  terrain  altitudes  from  the  lowest  point  to  the  highest.  The 
coloring  scheme  applied  to  the  SCENARIO  terrain  was  a  standard  for  physical 
geography,  ranging  from  deep  greens  to  light  browns  as  altitude  increases. 
This  scheme  was  used  because  it  was  easily  understood  and  implemented  by  the 
development  team.  Another  alternative  would  have  been  to  use  the  internal 
lighting  models  provided  with  the  Silicon  Graphics  computer.  This,  however, 
would  have  placed  additional  burdens  on  the  development  of  the  system  and  on 
its  operation,  since  the  terrain  coloring  would  need  to  be  recalculated  for 
each  display.  Because  the  team  felt  that  the  altitude  shading  was 
sufficient,  it  decided  not  to  pursue  the  use  of  lighting  models  at  the 
expense  of  remaining  time  required  to  implement  the  rest  of  the 
demonstration. 

Two  problems  eventually  surfaced  regarding  the  use  of  the  technique  for 
the  demonstration.  Upon  review  of  the  demonstration  it  became  apparent  that 
the  use  of  altitude-based  shading  presented  a  blurring  of  surface  features. 
The  purpose  of  the  technique  is  to  present  a  smooth  transition  of  shading  as 
altitude  changes.  Viewed  from  above  in  high  resolution,  the  viewer  is  able 
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to  determine  relative  terrain  features  without  the  use  of  3D.  Viewed  from  a 
low  angle  and  up  close,  however,  mountain  ridges  of  approximately  the  same 
height  were  the  same  color  and  blended  together. 

The  second  problem  was  a  consequence  of  the  terrain  data  itself.  The 
region  included  in  the  San  Luis  Obispo  sample  was  chosen  because  it  contained 
a  greater  range  of  altitudes.  Unfortunately,  one  corner  of  the  sample 
included  a  particularly  high  peak.  When  the  range  of  altitudes  was  computed 
and  the  spectrum  of  colors  assigned  to  specific  altitude  sub-bands,  the 
result  was  that  a  large  portion  of  the  high  altitude  colors  were  used  only 
for  the  high  peak,  leaving  the  vast  majority  of  the  terrain  in  similar  shades 
of  green.  Several  possible  improvements  were  discussed,  such  as  changing 
shading  of  the  terrain  as  it  receded  from  the  viewer  or  modifying  the  terrain 
data  to  reduce  the  altitude  of  the  peak.  The  lack  of  time  and  labor  hours, 
however,  permitted  only  limited  modifications  to  the  existing  system.  The 
algorithm  for  determining  the  color  of  a  particular  pixel  was  modified  to 
allow  for  some  variability  in  the  shade  of  color  assigned,  with  the  intent  of 
providing  a  sense  of  texture  to  the  terrain. 

In  addition  to  posing  unique  design  issues  relating  to  the  effective  use 
of  3D,  the  development  of  SCENARIO  provided  a  first-hand  opportunity  to 
explore  the  capabilities  and  constraints  of  stereoscopic  image  presentation. 
It  became  quickly  apparent  that  the  implementation  of  stereo  graphics  was  not 
simply  a  matter  of  generating  dual  images.  The  ease  with  which  stereoscopic 
depth  was  perceived  varied  markedly  for  different  people  as  well  as  for  a 
single  viewer  during  different  sessions.  In  general,  however,  it  was  noted 
that  the  ease  and  comfort  of  perception  of  the  stereo  environment  improved  as 
the  user  worked  on  the  system.  In  fact,  the  degree  to  which  depth  was 
perceived  increased  with  continuous  exposure.  The  problems  encountered  in 
the  use  of  the  stereo  display  were  essentially  the  inconsistency  and 
unpredictability  of  the  viewer  to  perceive  stereoscopic  depth  by  successfully 
fusing  the  stereo  images.  The  greater  the  intended  stereoscopic  effect  (as 
determined  by  the  adjustable  viewing  parameters),  the  more  pronounced  the 
problem  of  immediate  and  consistent  perception.  By  providing  user  control 
over  the  setting  of  the  viewing  parameters,  the  development  team  felt  that 
the  viewer  could  adjust  them  to  maximum  effect  during  the  session  and  the 
software  developer  could  experiment  with  the  capabilities  of  the  stereoscopic 
technology. 

The  SCENARIO  demonstration  software  provides  the  user  with  an  example  of 
3D  display  technology  applied  to  a  battle  management  application.  Project 
constraints  in  terms  of  team  experience  with  3D  development,  budget  and  time 
restrictions,  and  equipment  performance  capabilities  necessarily  limited 
aspects  of  the  implementation.  In  most  cases,  these  limitations  can  be 
overcome  by  further  development  of  the  system.  In  addition,  the  combined 
hardware  and  software  provide  a  powerful  and  expandable  stereo-equipped  3D 
graphics  workstation  for  continued  exploration  and  development.  The  SCENARIO 
source  code  contains  examples  of  many  aspects  of  stereo  programming  on  the 
system,  and  its  modular  structure  facilitates  both  enhancement  of  SCENARIO  as 
well  as  the  use  of  its  module  in  other  programs. 


36 


6.0  CONCLUSIONS 


A  potential  exists  for  achieving  real  and  significant  improvements  in 
the  performance  of  certain  battle  management  activities  as  a  direct  result  of 
the  use  of  three  dimensional  displays.  Primary  anticipated  benefits  are  a 
decluttered  screen  image  providing  greater  accuracy  and  ease  of  use  by  the 
viewer;  an  improved  accuracy  in  perceiving  the  mass,  surface  areas  and 
intersections  of  volumetric  forms;  and  an  increased  insight  into  the 
relationships  of  objects  as  they  interact  over  time.  C3I  battle  management 
functions  that  can  put  these  enhanced  capabilities  to  the  best  use  are  in  the 
areas  of  situation  assessment  and  operational  mission  planning,  particularly 
applications  which  involve  the  presentation  of  battlefield  dynamics  for 
purposes  of  evaluation.  These  situations  make  optimal  use  of  the  unique 
advantages  of  spatial  and  temporal  modeling  afforded  by  3D  displays. 

The  ability  to  realize  the  potential  benefits  of  3D  displays  is 
constrained  by  the  state  of  available  technology.  Factors  important  to  the 
successful  application  of  3D  technology  to  C3I  battle  management  tasks 
include  image  resolution,  full  color  capabilities,  real-time  processing  and 
display  speeds,  full  motion  parallax,  robust  multiviewer  support,  and  minimum 
requirements  for  user-worn  aids.  Techniques  that  rely  on  photographic  or 
computer-generated  imagery  typically  provide  the  resolution  required,  though 
photographic  systems  require  image  collection  instead  of  dynamic  generation 
and  typically  involve  processing  times  inappropriate  for  real-time  efforts. 
A  related  issue,  however,  is  the  display  volume  or  space.  The  SpaceGraph, 
for  example,  provides  a  very  compelling  image  partially  because  of  the 
resolution  achieved  in  a  relatively  small  display  volume.  Full  color  is 
currently  achievable  with  some  computer  graphics  systems.  The  demand  for 
real-time  display  capability  limits  the  effectiveness  of  several  techniques, 
although  in  many  cases  real-time  processing  is  a  function  of  the  underlying 
computer  power.  Preservation  of  full  motion  parallax  in  a  display  system  has 
been  achieved  in  only  one  commercially  available  system,  the  SpaceGraph 
vibrating  mirror  system. 

Several  techniques  allow  the  image  to  be  viewed  by  more  than  one 
observer,  but  the  problem  of  human  vision  variability  and  visual  fatigue 
affects  all  3D  methods  to  some  degree.  Studies  indicate  that  a  significant 
percentage  of  the  general  population  (between  15%  and  30%)  are  in  some  way 
stereoscopical ly  impaired.  Because  of  the  potentially  harsh,  often 
uncontrollable  conditions  under  which  the  targeted  user  must  be  able  to 
operate,  it  is  critically  important  to  minimize  the  negative  effects  of  the 
technology  on  the  user.  A  fieldable  system  should  be  equally  usable  by 
anyone  and  require  no  special  training  or  capabilities.  Ideally,  this  would 
include  the  requirement  for  user-worn  viewing  aids.  In  addition,  the  viewer 
must  be  able  to  tolerate  prolonged  use  of  the  system  with  a  minimum  amount  of 
effort  or  risk  of  visual  fatigue.  While  technological  research  continues  to 
spur  new  developments  in  3D  display  techniques,  very  little  directed  research 
into  the  human  factors  issues  associated  with  the  technology  has  been 
performed.  Those  marketing  the  new  technology  will  continue  to  determine  the 
direction  of  its  development  until  studies  are  performed  that  clearly  relate 
these  new  and  different  techniques  to  human  perceptive  abilities  and  identify 
their  effects,  both  positive  and  negative. 
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A  final  issue  in  the  commercial  development  and  eventual  deployment  of 
30  systems  is  the  lack  of  suitable  input  devices.  While  the  viewer  is 
receiving  three  dimensional  output  from  the  system,  he  has  no  easy  way  to 
move  a  cursor  or  otherwise  select  points  within  that  space.  Some  attempts 
are  being  made  to  fill  this  gap.  In  an  evaluation  of  3D  cursors  (3D 
trackball;  conventional  X,  Y,  yaw;  3D  mouse  with  selectable  XY,  XZ,  YZ 
planes;  solid  3D  thumbwheels;  planar  thumbwheels;  Z-axis  slider  with  X-  and 
Y-axis  thumbwheels),  the  fewest  positioning  errors  were  made  by  subjects 
using  the  planar  thumbwheels,  followed  by  the  Z-axis  slider  type  cursor.  The 
3D  trackball  and  mouse  performed  the  worst.  The  evaluation  concluded  that 
vector-oriented  cursors  were  better  than  planar  (XY,  XZ,  YZ)  or  free-space 
(trackball)  cursors,  though  none  of  the  devices  were  considered  to  be  very 
good. 


The  SCENARIO  demonstration  system  was  developed  to  demonstrate  the 
benefits  of  3D  to  a  specific  battle  management  task  area.  In  deciding  which 
display  technology  to  use,  commercial  availability  limited  the  choice  to  the 
SpaceGraph  vibrating  mirror  and  a  stereoscopic  shuttering  system.  The 
selection  of  the  stereoscopic  technique  was  based  in  large  part  on  the 
relative  commercial  maturity  of  the  product  and  of  the  supporting  hardware 
and  software  interfaces.  The  decision  to  purchase  an  implementation  based  on 
Silicon  Graphics  and  StereoGraphics  equipment  was  the  result  of  team 
familiarity  with  and  access  to  equivalent  equipment  as  well  as  cost 
restrictions.  The  alternative  was  to  purchase  the  SpaceGraph  and  attempt  to 
develop  the  necessary  support  software  while  learning  how  to  program  the 
device  and  develop  an  acceptable  demonstration  with  the  technological 
constraints  of  the  display  device.  All  of  these  decisions  were  affected  by 
the  limited  effort  funds  and  relatively  short  amount  of  time  in  which  to 
acquire,  install  and  develop  a  3D  demonstration  system.  As  a  result,  the 
team  favored  the  most  conservative  option,  which  resulted  in  providing  the 
client  with  a  powerful  and  expandable  graphics  workstation  with  a 
stereoscopic  display  capability. 

The  demonstration  software  was  developed  around  a  mission  planning 
scenario  mutually  agreed  upon  by  the  study  team  and  the  government.  While 
the  actual  development  of  software  which  supported  the  stereo  system  was  not 
much  more  complex  than  software  for  conventional  displays,  it  did  require  an 
added  effort.  As  the  system  was  being  developed,  many  lessons  were  learned 
by  observing  the  stereo  image  and  evaluating  its  effect.  Some  decisions, 
such  as  originally  deciding  to  project  all  images  behind  the  plane  of  the 
screen  (positive  parallax),  seemed  obvious  until  the  effects  of  viewing 
objects  in  front  of  the  screen  (negative  parallax)  were  experienced.  In  an 
effort  to  accentuate  the  3D  features  of  the  display,  decisions  were  made 
which  were  later  recognized  as  having  reduced  the  effectiveness  of  the 
display  by  removing  valuable  monocular  cues.  Most  design  decisions,  such  as 
committing  to  a  terrain  coloring  scheme  based  on  altitude  rather  than  using 
available  system  lighting  models,  were  based  on  an  estimate  of  the  available 
labor  hours.  Others,  like  the  default  terrain  resolution  settings,  reflect 
computational  restrictions  of  the  system  speed. 

Building  SCENARIO  was  a  3D  learning  process  as  well  as  a  software 
development  one.  The  resulting  program  suffers  from  some  deficiencies  that 
can  be  corrected  as  experience  with  the  system  Is  gained.  As  delivered, 
however,  the  SCENARIO  code  provides  the  user  with  the  capability  to  explore 
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the  use  of  a  stereoscopic  display  in  support  of  a  particular  C3I  function. 
As  images  and  scenes  are  discovered  or  generated  that  demonstrate  certain 
characteristics  of  the  technology,  they  can  be  saved  for  later  display  and 
examination.  Access  to  the  source  code  allows  the  system  to  be  enhanced  as 
desired,  using  either  existing  routines  that  support  the  stereo  display 
equipment  as  working  examples  or  incorporating  newer  and  more  powerful 
algorithms. 

In  summary,  given  the  current  state  of  the  technology,  three  dimensional 
displays  can  provide  the  viewer  with  a  perceptual  reconstruction  of  three- 
space  that  is  not  achievable  otherwise.  The  use  of  3D  may  be  important  when 
monocular  cues  alone  provide  ambiguous  depth  information.  Stereoscopic  3D 
can  resolve  those  ambiguities,  though  depending  on  the  technology,  it  may  be 
at  the  potential  risk  of  some  viewer  discomfort. 


7.0  REFERENCES 


This  section  lists  reference  material  used  in  the  course  of  this  study. 

Beaton,  Robert,  DeHoff,  Richard  J.,  Weiman,  Novia  and  Hildebrandt,  Peter  W. 
(1987).  An  Evaluation  of  Input  Devices  for  3-D  Computer  Display  Workstations 
(SPIE  Vol.  761  True  3D  Imaging  Techniques  and  Display  Technologies).  Imaging 
Research  Laboratory,  Tektronix  Laboratories,  Beaverton,  Oregon. 

Beaton,  Robert  and  Weiman,  Novia  (1988).  User  Evaluation  of  3-D  Cursors  (SPIE 
Conference),  Tektronix  User  Interface  Laboratory,  Beaverton,  Oregon. 

Bevan,  W.  and  Steger,  J.  A.  (1971).  Free  recall  and  abstractness  of  stimuli. 
Science,  172,  pp.  597-599. 

Bower,  G.  H.  (1972).  Mental  imagery  and  associative  learning.  In:  L.  W. 
Gregg  (Ed.),  Cognition  in  Learning  and  Memory.  New  York:  Wiley. 

Brooks,  L.  R.  (1968).  Spatial  and  verbal  components  of  the  act  of  recall. 
Canadian  Journal  of  Psychology,  22,  pp.  349-368. 

Byrne,  B.  (1974).  Item  concreteness  vs.  spatial  organization  as  predictors 
of  visual  imagery.  Memory  and  Cognition,  2,  pp.  53-59. 

Finegold,  Lawrence  S.,  Asch,  Anthony  J.,  and  Flaugher,  James  G.  (1984). 
Simulated  Three-dimensional  Graphics  Training  Display  for  Air  Weapons 
Controllers.  Report  No.  AFHRL-TP-83-62,  Air  Force  Systems  Command,  Brooks 
Air  Force  Base,  Texas. 

Fisher,  S.  S.,  McGreevy,  M.,  Humphries,  J.  and  Robinett,  W.  (1986).  Virtual 
Environment  Display  System  (ACM  1986  Workshop  on  Interactive  3D  Graphics). 
Aerospace  Human  Factors  Research  Division,  NASA  Ames  Research  Center,  Moffett 
Field,  California. 

Kneale,  Dennis  (1988).  Into  the  Void.  The  Wall  Street  Journal,  Vol.  CCXI, 
No.  7,  Page  1,  January  12,  1988. 


39 


Kosslyn,  S.  M.,  Ball,  T.  M.  and  Reiser,  B.  J.  (1978).  Visual  images  preserve 
metric  spatial  information:  Evidence  from  studies  of  image  scanning. 

Journal  of  Experimental  Psychology:  Human  Perception  and  Performance,  4,  pp. 
47-60. 

Kramer,  Keiko  (1988).  Applications  and  Benefits  of  3-D  Displays.  Project 
correspondence,  Honeywell  SRC,  Minneapolis,  Minnesota. 

Laur,  David  and  Alexander,  Kirk  (1988).  Evolution  of  the  Princeton 
Visualization  Tools  (1988  Princeton  symposium  on  Visualization  in  Scientific 
Computing).  Interactive  Computer  Graphics  Laboratory,  Princeton  University, 
Princeton,  New  Jersey. 

Leeper,  R.  (1935).  A  study  of  a  neglected  portion  of  the  field  of  learning 
--  the  development  of  sensory  organization.  Journal  of  Genetic  Psychology, 
46,  pp.  41-75. 

Lentz,  Paul  and  Goss,  Michael  (1988).  C3I  Functions  and  Taxonomy.  Project 
correspondence.  Merit  Technology,  Plano,  Texas. 

Lipton,  Lenny  (1988).  Displays  Gain  Depth.  Computer  Graphics  World,  March, 
1988. 

Lipton,  Lenny  (1988).  3Display  Programmer's  Guide.  StereoGraphics 
Corporation,  San  Rafael,  California. 

Mandler,  J.  M.  and  Parker,  R.  E.  (1976).  Memory  for  descriptive  and  spatial 
information  in  complex  pictures.  Journal  of  Experimental  Psychology:  Human 
Learning  and  Memory,  2,  pp.  38-48. 

Mandler,  J.  M.  and  Ritchey,  G.  H.  (1977).  Long-term  memory  for  pictures. 
Journal  of  Experimental  Psychology:  Human  Learning  and  Memory,  3,  pp.  386- 
396. 

Meacham,  G.  B.  Kirby  (1986).  Autostereoscopic  Displays  -  Past  and  Future 
(SPIE  Vol.  624  Advances  in  Display  Technology  VI).  Strategic  Technology 
Inc.,  Cleveland,  Ohio. 

Phillips,  T.  E.  (1984).  Stereoscopic  and  Volumetric  3-D  Displays:  Survey  of 
Technology  (NOSC  Technical  Report  No.  946).  Naval  Ocean  Systems  Center,  San 
Diego,  California. 

Piantanida,  Thomas  (1988).  Taxonomy  of  Stereoscopic  Display  Systems. 
Project  correspondence,  SRI  International,  Menlo  Park,  California. 

Piantanida,  Thomas  (1988).  Application  of  Stereodisplays  to  C3I.  Project 
correspondence,  SRI  International. 

Ramacbandran,  Vllayanur  S.  (1988).  Perceiving  Shape  from  Shading. 
Scientific  American,  August,  1988. 

Relslng,  John  (1988).  Focus  on  Research:  3D  Stereo  Display  Formats.  ASTG 
Newsletter,  HQ  USAFA/DFBL,  Colorado  Springs,  Colorado. 


40 


Robitaille,  Stephen  (1987).  See  the  future?  He's  touched  It.  San  Jose 
Mercury  News,  November  30,  1987. 

Ross,  J.  and  Lawrence,  K.  A.  (1968).  Some  observations  on  memory  artifice. 
Psychonomic  Science,  13,  pp.  107-108. 

Shepard,  R.  M.  (1967).  Recognition  memory  for  words,  sentences  and  pictures. 
Journal  of  Verbal  Learning  and  Verbal  Behavior,  6,  pp.  156-163. 

Standing,  L.,  Conezio,  J.  and  Haber,  R.  N.  (1970).  Perception  and  memory  for 
pictures:  Single-trial  learning  of  2560  visual  stimuli.  Psychonomic 

Science,  19,  pp.  73-74. 

Thompson,  Steven  L.  (1987).  The  Big  Picture.  Air  &  Space  magazine,  April/May 
1987,  Smithsonian  Institution,  Washington,  D.  C. 

Watkins,  Robert  (1988).  Through  the  Looking  Glass.  Computer  Graphics  World, 
January  1988. 

Way,  T.  C.,  Hornsby,  M.  E.,  Gilmour,  J.  D.,  Edwards,  R.  E.  and  Hobbes,  R.  E. 
(1984).  Pictorial  format  display  evaluation  (Report  No.  AFWAL-TR-84-3136) . 
Air  Force  Wright  Aeronautical  Laboratory,  Flight  Dynamics  Laboratory,  Wright- 
Patterson  Air  Force  Base,  Ohio. 

Way,  Thomas  C.  (1988).  Stereopsis  in  Cockpit  Display  -  A  Part-task  Test 
(Proceedings  of  the  Human  Factors  Society  32nd  Annual  Meeting).  The  Boeing 
Company,  Seattle,  Washington. 

Williams,  T.  (1988).  Input  technologies  extend  the  scope  of  user 
involvement.  Computer  Design,  March  1,  1988. 

Wollen,  K.  A.,  Weber,  A.  and  Lowry,  D.  (1972).  Bizarreness  versus 
interaction  of  mental  images  as  determinants  of  learning.  Cognitive 

Psychology,  3,  pp.  518-523. 

Wright,  Robert  (1987).  The  Information  Age:  Virtual  Reality.  The  Sciences 
journal,  November/December  1987. 

Zenyuh,  John  P.,  Reising,  John  M.,  Walchli,  Scott  and  Biers,  David  (1988).  A 
Comparison  of  a  Stereographic  3-D  Display  versus  a  2-D  Display  Using  an 
Advanced  Air-to-air  Format  (Proceedings  of  the  Human  Factors  Society  32nd 
Annual  Meeting).  Flight  Oynamics  Laboratory,  Wright-Patterson  Air  Force 
Base,  Ohio. 


41 


